The evolving role and skills related to quality assurance

There was a time when quality assurance (QA) was nothing more or less than the last safety net before product delivery. Developers churned out code, quality assurance teams flagged anomalies, and the process reiterated. The value of QA teams was measured in terms of the number of bugs they found, and the value of a developer by the number of bugs per line of code.

Just imagine, then, the endless arguments between QA and development teams. QA would claim they’d found a bug, only to have the developers (of which I was a part at one time) argue that it was a “design intent”, so as to avoid losing points. Seems crazy now, but that’s the way it was.

Of course, as time went by, this climate of confrontation yielded to one of greater mutual understanding, though I wouldn’t go so far as to call it cooperation.

But back to today’s situation. Development best practices require the implementation of a continuous quality control process. This means that every iteration should be followed by non-regression testing. Since these tests are cumulative, they become more and more onerous as development progresses, until the QA team spends more time running non-regression tests than integration tests. The worst part is that sooner or later, the QA team will decide to stop running non-regression tests in order to secure more time for feature validation.

Yet software developers agree: the vast majority of bugs (over 80%) are introduced in the codebase at the module design and editing stages. When too much time is spent on non-regression testing, bugs are only found at the integration testing phase, i.e. late in the process, and sometimes even too late.

Keep in mind that the cost of correcting a bug increases by a factor of 30 if detected only at the end of integration testing, and by a factor of 80 if detected at the production stage.

Ideally, “shift-left testing” methods should be implemented in order to detect bugs as soon as they crop up, i.e. as the code is written. The idea is to narrow the gap between the two peaks in the graphic above (from the presentation “Pas d’agilité sans qualité”, “No agility without quality”, given by François Boretto from Askida at Agile Tour Montreal 2018).

And this is where quality assurance comes in: how can this “shift-left” be implemented?

First of all, QA analysts must be involved upstream, at the scenario development stage. Analysts must be open to, and conversant with, several different technologies, and must be involved and embedded in the development team. Management must support and encourage this approach. Further, QA teams must be released from tedious non-regression tests.

In other words, what is required is a DevOps-familiar team that masters the concept of continuous development and integration. Out with basic testers, and in with QA analysts with the skills required to assemble a list of critical functionality tests and to automate these tests. Scripts can be executed at the development stage and then again at each iteration as a non-regression test. Now, there is just one thing to consider, and that is the development of functional tests. Then, non-regression tests execute automatically at stated intervals. And it goes without saying that analysts must have the skills required to manage and maintain a test environment themselves.

In other words, the skill set required for QA is converging with that required for developers. Not only must analysts be able to test, but they also need some knowledge in coding. Chances are, manual functional tests being performed at the time of code submission will become the exception rather than the rule. But as not everything can, or should, be automated, there will always be a (marginal) need for manual testing. Consequently, testers with no automation skills are bound to become irrelevant.

My take-away from the literature I’ve parsed (example) is that henceforth, QA teams must develop their test automation and script writing skills. By evolving in this direction, QA teams will be able to perform functional tests earlier in the development cycle without compromising the frequency of non-regression tests.

Jean Godard, Ing.