Why software developers need a QA gate

Benji Shults
Sep 9, 2018 · 5 min read

Because we are human.

Before we get to that…

What is a QA gate?

A typical engineering process begins with the business defining the problems that the product is supposed to solve. These requirements are given simultaneously to the developers and to the quality assurance (QA) engineers.

Then the product moves through several environments or stages. Commonly:

  1. Development. The development environment is a sandbox where developers can work and test prior to the product going to higher environments. We expect things to break in this environment from time to time.
  2. QA. This environment is identical to production but is not used by customers. QA engineers use this environment for testing. This is our last chance to catch issues prior to them affecting customers. When QA engineers find a problem, they push the product back to the developers.
  3. Production. This is the environment that customers use. This is the environment where we do not want to find problems.

A typical software delivery process will include a gate between each of those environments.

Before it goes to the QA environment, developers must approve the product (perhaps through an automated process or perhaps manually.)

Before it goes to the production environment, QA engineers must approve the product (perhaps through an automated process or perhaps manually.) This is called a “QA gate”.

Why software developers think we don’t need a QA gate

Here is, perhaps, the best-case scenario for arguing that developers do not need a QA gate.

  1. Before writing code, we write acceptance tests. These tests run on a schedule in each environment. These tests test our application from the outside, hitting its endpoints and validating that it behaves properly. These tests also run on the new application in each environment prior to it taking “real” traffic in that environment.
  2. Before writing code, we write automated integration tests which test how our new code will interact with other components in our application.
  3. Before writing code, we write automated unit tests that test the details of just the bit of code we are working on.
  4. Our team is doing pair programming so there are always at least two sets of eyes on the code while it is being developed.
  5. Before code is merged into the main branch, a third developer reviews and challenges it.

If you don’t have all that in place, that is enough of a reason to believe that you need a QA gate.

But since we do have all that in place, we might argue that we don’t need QA.

Here are our arguments:

  • The problems that QA is supposed to solve are already solved by our automated tests and process.
  • QA engineers sometimes write bad tests that fail spuriously.
  • A QA gate slows us down…
  • and defeats the purpose of continuous integration and continuous delivery (CI/CD.)

Why we are wrong

Developers are human. I don’t mean you or me. I mean other, imperfect developers. They are human.

And we need a process that works for humans.

Humans make mistakes. Humans have blind spots. Humans have blind spots blocking them from seeing some of their blind spots.

The problem QA solves

The QA engineers take the requirements and come to understand what they mean and how they can be tested. They do this without being influenced by the developers’ interpretation of the requirements and how the solutions are built.

Being human, developers may misinterpret the requirements and all the developers may agree in the misunderstanding of the requirements because they are social creatures.

Being human, developers may make compromises to save costs and all the developers may agree on these compromises.

Being human, developers may fail to communicate perfectly with the folks representing the business and the problem that is to be solved.

For all these reasons, we need QA.

Addressing the arguments

Let’s address the arguments we made above.

The problems that QA is supposed to solve are already solved by our automated tests.

The problem QA solves is the fact that developers make mistakes. Sometimes, they make mistakes in how they interpret the requirements. Automated tests don’t change any of that.

QA engineers sometimes write bad tests that fail spuriously.

Yes, they do. The reason they sometimes write bad tests is that they are also human. Like us. And we sometimes write bad tests, too. And we’re back to needing QA.

Developers sometimes write bad tests that pass spuriously!

And, yes, sometimes QA engineers make other mistakes and sometimes the process is bad and …

In the next section, I’ll address the fact that sometimes there really are problems with our QA process.

A QA gate slows us down…

This is a legitimate argument. It may slow down the process of getting the product to customers.

If your tests really are perfect and cover everything, then the delay should be very small. If your tests are not perfect (they aren’t) or do not cover every possible case (they don’t), then the delay is probably worth it.

… and defeats the purpose of continuous integration and continuous delivery (CI/CD.)

I think it does not. I don’t believe there is a conflict. But I won’t argue that here because I suspect we have different ideas about the purpose of CI/CD.

Supposing for a moment that a QA gate does defeat the purpose of CI/CD.

Then the business (and not developers) will need to decide which is more important. It is up to the business to decide whether the delay is worth it. It is up to the business to make these informed decisions.

Engineering businesses have been insisting on a QA gate for a long time. This has to mean something.

If they decide we need a QA gate, then we are good enough engineers to come up with a good CI/CD pipeline that includes a QA gate.

Problems remain

Yes, sometimes QA write bad tests.

Sometimes QA engineers understand the requirements wrong.

Sometimes there is not good communication between business and developers and QA.

Sometimes QA engineers are just too darn slow.

“Sometimes a roof leaks” is not an argument for doing away with roofs. But it is a very good argument for fixing the roof.

I tend to believe that improved process and improved communication can reduce the negative impact of the fact that so many developers and QA engineers are human beings. (Not us! Not us! Them!)

It is not perfect and it will never be perfect. Sometimes problems come up that no amount of process can protect you from. For the rest of the problems: keep improving the process.

Conclusion

In my opinion, there is no (or there should be no) conflict between a QA gate and other valuable development practices.

It is the business representatives that should make an informed decision about what processes and gates are dispensable or what conflicts need to be resolved by adjusting process.

Have you heard (or do you have) other arguments against having a QA gate? Have I completely missed something big? Let me know (respectfully and civilly) in the comments.

Benji Shults

Written by

Staff Software Engineer at SmartThings with a PhD in Mathematics and Artificial Intelligence from UT-Austin

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade