Featured Product
This Week in Quality Digest Live
Lean Features
Shiela Mie Legaspi
Set SMART goals
Gleb Tsipursky
Belief that innovation is geographically bound to office spaces is challenged by empirical evidence
Jamie Fernandes
From design to inspection to supply chain management, AI is transforming manufacturing
James Chan
Start the transition to preventive maintenance
Mark Rosenthal
The intersection between Toyota kata and VSM

More Features

Lean News
New video in the NIST ‘Heroes’ series
Embrace mistakes as valuable opportunities for improvement
Introducing solutions to improve production performance
Helping organizations improve quality and performance
Quality doesn’t have to sacrifice efficiency
Weighing supply and customer satisfaction
Specifically designed for defense and aerospace CNC machining and manufacturing
From excess inventory and nonvalue work to $2 million in cost savings
Tactics aim to improve job quality and retain a high-performing workforce

More News

Boris Shiklo


How QA Consulting Saved a Software Project

The true story of an endangered project

Published: Thursday, May 9, 2019 - 11:03

About 10 years ago, software testing was perceived as the only possible quality assurance (QA) measure for software, according to the World Quality Report 2018–2019. However, QA has since outstepped these boundaries. The QA process now implies that all stakeholders have a direct interest in software quality during the entire project life cycle. But how should you establish such a comprehensive QA process?

Let’s consider what obstacles you may face while setting it up and explore a real-life example of how software QA consulting can solve QA problems.

Why QA consulting?

An established QA process should build in defects detection right from the project planning stage. QA teams should dwell on bugs prevention rather than just finding them, and regularly measure testing effectiveness using clear quality metrics. Project managers, business analysts, and developers should do their share in establishing a solid QA process.

For example, project managers should allocate sufficient time and human resources for testing, and establish an efficient working process promoting seamless communication between stakeholders. Business analysts should involve QA managers as early as the requirements-gathering stage to ensure that requirements are complete and stated in a way appropriate for future testing. Developers shouldn’t ignore unit testing and code reviews.

Many companies find they can’t establish such an all-encompassing QA process using only in-house efforts. Reasons for this failure are various:
• Underestimation of QA importance among stakeholders
• Hectic attempts to improve QA
• Lack of time to get an all-round view of the existing QA complications
• Inability to uncover the root causes of evident QA problems
• Non-QA issues such as meeting projects’ deadline and budget

Other companies need QA help when their business expands, the workload increases, or they’ve just introduced an in-house QA team.

In all cases, QA consulting is applicable because it can help either with a general QA maturity ramp-up or with a specific project that came to a QA deadlock.

The best way to understand how QA consulting may be helpful is to consider a real-life example of its implementation. This is a true story of an endangered project, whose team got lost in numerous QA issues, and of a QA consultant who saw the core of the QA problems and offered an effective solution.

How the project started

A small-sized software development company was commissioned by a large multinational enterprise to supplement its customer relationship management system (CRM) with a custom-developed additional application to introduce complex, industry-specific workflows, and to integrate that CRM with its existing enterprise resource planning and warehouse management solutions.

The team of 16 developers and five overseas test engineers got down to the project. The agile methodology of software development was chosen for the project so that the customer could regularly receive the functioning parts of the CRM.

How the project went bad

Test engineers stepped in only after two iterations had passed, so there was already a functioning part of software to be validated even as another iteration was in progress. The test team was snowed under creating and executing test cases. The team was also discouraged by poor code quality, and by developers who neglected essential unit tests and didn’t inform them about already-fixed bugs.

In their turn, the development team blamed the test engineers for incoherent bug reports that complicated the bugs’ reproduction and slowed down the fixing process, thus leading to missed iteration deadlines. As test engineers found more bugs, the process went from bad to worse.

Because the developers were trying to cope with regression errors appearing in every new build, they stopped checking in code in the source-code management tool. This resulted in numerous bugs that still needed to be fixed and tested, development teams not meeting iteration deadlines, a lot of test cases remaining unexecuted, and some requirements not being covered by test cases.

Enter QA consulting

Facing such a QA havoc resulting from poor development and testing results, the project manager turned to QA consulting to save the project.

A QA consultant started with the investigation and analysis of the current situation. To gain insight into the project’s pitfalls, he held interviews with the stakeholders, and acquainted himself with the project documentation and requirements specification. To objectively evaluate the maturity level of the existing QA process, the QA consultant used test maturity model integration (TMMi). The assessment revealed that the project’s QA maturity was at the initial level. Only debugging activities were carried out, and the testing documentation was incomplete.

Then, the QA consultant proceeded to do a root-cause analysis. Among the most acute reasons for QA getting into such a gridlock was the lack of QA awareness among the stakeholders, poor alignment between the development and test teams, a lack of established communication channels, and a lack of test management tools. Both the developers and test engineers perceived QA solely as a bug-detection function. The interviews showed that the developers believed that everything bound to testing was not their business—even unit testing and code reviews. The test engineers believed that the QA process would have been assured if developers had stored source code properly and conducted unit testing.

The QA consultant prepared a detailed report that explained the results of the QA maturity measurement, outlined the causes of the actual problems, and provided a number of solutions. The QA professional presented and discussed the report with the project manager, and proposed an action plan. The following QA improvement measures were implemented with the QA consultant’s assistance:

Meetings with the developers were held to explain the need of sticking to QA aspects in coding (for example, code readability, testability, robustness) as well as the necessity of regular code reviews and unit testing.

A video conference was organized to explain to test team members that they should actively participate in creating user stories and come up with relevant intermittent improvements in the testing quality.

Regular communication was established within the project team. Developers and test engineers posted their topical daily tasks each morning in a shared chat to coordinate and prioritize testing efforts, and participated in weekly video conferences to keep up with the pace of the project.

Also, the QA consultant advised on a unified continuous integration and a bug-tracking tool that supported screen-recording functions. This way, test engineers could save time by not writing endless detailed bug descriptions, and developers spent less time on defects reproduction.

How the project fared afterward

In three weeks, the QA consultant checked the relevant testing performance results after two finished iterations. Among the measured KPIs were:
• The ratio between the number of bugs found and defects fixed during the iteration
• The adherence to the deadline
• The number of executed test cases (with regard to their complexity)
• The number of requirements not covered with test cases

The consultant shared the QA efficiency assessment results with the whole team. The improvements were evident. The percentage of bugs fixed during the two iterations in question were 54 percent and 87 percent, respectively, compared with 18-percent before QA consulting. Teams managed to meet the goals of both iterations without delays. The number of executed test cases nearly doubled, and all iterations’ requirements were covered with test cases.

So, is this how the story ended? No. The QA consultant stayed in touch with the project manager for some time afterward, advising on how to face the minor QA troubles that arose during the rest of the project.

Lessons learned

An efficient QA process requires sufficient time and organizational efforts. The whole project team should contribute to it and understand that software quality is of utmost long-run importance. And if it comes to QA roadblocks, a handy and functional way out is to turn to QA consultants. They are ready to step into the whirl of QA problems, investigate the root causes, propose actionable solutions, assist with their implementation, and pave the way for stable software QA.



About The Author

Boris Shiklo’s picture

Boris Shiklo

Boris Shiklo, CTO at ScienceSoft, is responsible for the company’s long-term technological vision and innovation strategies. Under his supervision, the company’s development team has successfully fulfilled complex projects of over 80,000 man-hours in Healthcare, Banking & Finance, Retail, Telecommunications, Public Sector and other domains. Boris Shiklo has a solid background in IT consulting, software development, project management and strategic planning.