Testing is Software Sin-Eating

This isn’t how it should be. But this is how it is — in my experience, and in my observation.

Software has problems. Programmers make mistakes. People miscommunicate. Releases get delayed, and people’s needs change. Software, sometimes, fails to serve the needs of its stakeholders.

There are solutions to all of these problems. Programmer mistakes? Pair programming. Continuous integration. Miscommunication? User research. Product manager acceptance testing. People’s needs change? Build the smallest thing that could possibly work, then iterate. The inevitability of error? Cheap, regular deployments. Automated rollback processes.

It goes on. There are more practices — real, useful, humane practices— than can be mentioned here.

But these practices are hard to adopt. They take skill. They take experience. They take, in many cases, a willingness to let go of small power, a willingness to shed false security. They take, especially, humility.

So instead, software stakeholders turn to magic. They buy tools. They pay consultants. They invest in “Best Practices.” They hire testers, to find bugs, and to reassure themselves and their masters that the software is “high quality.”

Then starts the sin-eating. Because even if these are excellent testers, clear-eyed seekers of truth, they don’t have access to real power. They can, at best, advise and explain. But they are not trained to diagnose and address organizational dysfunction. They are not equipped to make real changes.

And as much as we, as excellent testing practitioners, reject responsibility for “quality,” when stakeholders expect us to take on that mantle, if we want to keep working for them, then we must take it on. And sometimes we do.

So our stakeholders come to us, and they ask, “Have you found all the bugs? Is it ready to release?” and we say, “Well I did this testing and that testing, and I haven’t found anything.”

And they say, “Yes, but is it Ready to Release?” and we say, “Probably?”

(These conversations with stakeholders are real conversations that I have actually had. I am not proud of my part in them, but they happened.)

We are given responsibility for software quality, without any power over it. We do not reject that responsibility, because we like having jobs, and we are tired of being told that we are arguing over “semantics,” and we like finding problems in software. We are willing to put up with the boring parts of the job, and the parts that don’t feel right to us, in order to find problems in software.

So we eat sins. We find enough bugs, early enough, for our stakeholders in the organization go to their masters and say, “Behold! We have hired Testers and they are Good. We have Best Practices. We care about Quality. We are not responsible for the bugs that do make it to production. We do not have to change.”

And their masters say, “Yes, but there are still Bugs. We would like fewer of those. Or you will be fired, and your families will rot with the wolves.”

So our stakeholders come to us, and they ask, “Did you run all the tests? Have you followed best practices? Do you have a spreadsheet we can give to our masters to prove that were are making High Quality Software, and that we did Due Diligence, and that any bugs that are still in it are Not Our Fault?”

And we ask, “Should we keep ignoring those random crashes?”

And our stakeholders say, “Yes, the users can just restart the program.”