We explain how Google approaches quality without a separate QA department, and when testers are truly needed.
Why using testers in complex environments harms the product 15.2.2023 It is commonly believed that a serious IT company cannot do without a testing department. Today we will find out whether that is true.
Why does this setup not benefit the developer, the tester, or the product?
Why Google Has No Testers To test or not to test? When Testers Are Needed
Article published in IT-world magazine Reading time: 6 min.
It is commonly believed that a serious IT company cannot do without a testing department.
By applying the principle of division of labor, testers catch bugs more effectively, free up developers, and improve both product quality and value.
We are used to the Q.A. Passed label on various electronics and expect quality control to work the same way for IT products.
But development is not an assembly line.
Global experience, including kt.team's experience as an IT integrator, shows that in complex environments evaluation criteria are defined for each feature, which clearly goes beyond the competence of ordinary testers. Bringing them in blurs product value, raises cost and development time, and creates conflicts inside the team because responsibility is divided incorrectly.
I will break down in detail why this happens.
"Complicated environments" is a concept from the Cynefin framework, which divides all environments into four types: simple, complicated, complex, and chaotic. In linear (simple and complicated) environments, testers only need to follow the procedure strictly.
There, everything is clear: it is enough to compare plan and actual, ring out the components, or plug the device into an outlet.
The test result is unambiguous: the product either complies with the procedure or it does not.
There is no final outcome or fixed path to reach it.
There is no standard for acceptable or excellent results, only a direction for growth.
Typical testing evaluation criteria do not match the fundamental principles of Agile, which are most effective in complex environments. In a complex environment, responding to change and feedback comes first, not procedure.
Once upon a time in an IT company...
Both are put in a situation where the quality of the result takes a back seat and conflicts become inevitable. Here are the factors that trigger them.
It is bad when developers start thinking that their job is to deliver code rather than value. In the example above, with developer Petya and tester Vitya, neither thinks about value: the developer thinks about the number of features, and the tester thinks about the number of bugs. Business interests remain outside the process.
The requirements for the result change in every Agile cycle and for every feature. To avoid conflicts, you need to write evaluation criteria for each feature. That leads to a massive increase in bureaucracy and in the time needed to create the procedure. Both the developer and the tester will have much less time left for the product.
Run a small experiment described in the book "Focus."
First write "mama washed the frame" sequentially, then try writing the words in parallel: "m... m... r...", "ma. wa. fr." and so on.
Which option will take more time and effort? It is the same in development. When
Petya got bugs reported for task X ("ma."), while he is already deeply immersed in task Z ("ra.").
He had to remember his code, finish it up, or challenge the fixes, and then return to the current task again.
He will spend more time on context switching than on the task itself.
When a developer is not thinking about the errors or difficulties a user may encounter, they cannot anticipate them. So developer Petya, having completely delegated to his tester the task of thinking about how the user will interact, loses touch with reality while Vitya tries to look at the feature through the user's eyes.
Looking at MAANG companies - Meta* (banned in the CIS), Apple, Amazon, Netflix, and Google - we see that the idea of assembly-line testing is treated as outdated.
Industry leaders do not believe that bringing in testers by default adds meaningful value to software. For example, in the book "How Google Tests Software"
The authors point out that Feature Developers have everything they need to deliver quality value independently, and so-called Software Engineers in Test are outdated. Google writes about product IT companies, but the same logic also applies in service companies working in Agile. In Agile practices, testers do not exist as a separate role. For nonlinear work, the Cynefin framework recommends frequent value delivery and continuous feedback from production.
Accordingly, you need a single center responsible both for delivering value and for receiving that feedback - the developer. As soon as they start sharing responsibility with a tester, they inevitably lose focus.
Based on kt.team's experience, the optimal process setup is one where the developer:
Pair programming Test-driven development (tests are written by developers before code) Continuous integration Refactoring, and more.
This is a practical, proven way to avoid a large-scale failure. How did we get there?
We tested different ways of working with and without testers.
More than a year ago, based on experiments we had conducted, we removed the testing layer from nonlinear tasks.
Instead, we actively use extreme programming techniques: think about value first, not code; cover code with tests; and focus on getting feedback as quickly as possible through logging and dashboards. One more practical observation: to improve product quality, you need to make the most of different kinds of feedback.
I will not claim that testers are never needed in complex environments. There are situations where a team cannot do without Q.A.
For example, when building a mobile app.
There are many platforms and device variants on which the application must work correctly.
A developer can test it on several major platforms and devices, but running it across all of them using only the developer's time makes no sense. It is better to involve Q.A., especially if special hardware is used that has no software emulation, such as rare controllers from lesser-known manufacturers.
Bringing in testers here is justified: responsibility for product value remains with the developer, while testers are responsible for catching bugs when the system is rolled out across rare device variants.
Do not dump testing for popular browsers on Q.A.: functional tests handle these tasks much more efficiently. Besides, this is not 2010, and there is no problem if a button shifts in some version of a rare browser.
And automated tools for pixel-level DOM analysis for any browser are easy to access.
In this case, the company does not need an ordinary bug hunter, but a manager who thinks about broader goals and builds testing processes around them. They define testing areas, certify teams for quality, develop and implement test design standards, and set up processes for collecting and responding to user feedback.
There are niche areas of development that are tightly regulated by legal requirements or special security constraints.
Even in the banking sector, only a small part of the work falls into this category, and it is handled only by highly technical megacorporations.
These are companies with many systems working as one across all continents and operating under the same legal and other requirements.
Testers in such companies check compliance with regulations and adherence to unified standards.
Delivering value is not part of their responsibility: their job is to regularly review code and identify a specific vulnerability.
When they find a weak point, they add part of the automated test into the code that the team must then maintain.
In their work, it helps greatly when the company already has an established culture and a benchmark for evaluating developers' expertise.
