Testing vs. Checking — so what?

If you are curious or bewildered by the discussion about “testing vs. checking”, this post is for you. I read the article about testing versus checking by James Bach and Michael Bolton. At first I thought “so what?”. Then I thought it was some instance of dualism. Now I know better. And it even makes sense.

If you are in the testing community, you probably know the discussion about testing versus checking. Roughly speaking, the idea behind “testing vs. checking” is that checking is a purely mechanical task, where in contrast, testing can only be performed by humans. Since the distinction was propagated by James Bach and Michael Bolton, it got pretty famous in the testing community.

When I first read about the distinction, I couldn’t wrap my head around it. I thought “so what, we still have to do both”. Like, what is the benefit of the distinction, other than hairsplitting. What value does it add to the discussion? Calling things differently doesn’t change facts in the real world, so why would anyone bother? And I was not the only one who thought so:

Then I recognized that the distinction is all about what a machine can’t do and never will be able to do. Well, I know a few things about AI, and I don’t think there is anything that a machine will never be able to do. So I thought, this is some weird instance of dualism. Again, I was not the first one to think that. Nor the second one.

This is the point where I lost interest and shrugged it off to be some weirdo tester eccentric thing. But when I was at TestBash, people I respected made a point about that distinction. So I went back to that article and thought “what did I miss”?

What lies at the heart of the discussion

Turns out I did indeed miss something.

In most companies where I have been, testing was regarded as “low value”. Testers were mostly students, or even worse, people that had failed as developers. They “clicked through the application”, usually following a given script, and manually checked what they were told to check. There is nothing challenging or respectable in the task the way it is performed by most companies. This usually also reflects the stand of those companies on the topic of software quality: as something that is not that important, something that can be added later if the time and budget allows it. These “testers” are (mis-)used as “clicking machines”. And this is the sort of “manual automation” that can easily be outsourced, and often is.

But there are people and companies that put the quality of the software at the center of their effort. There is a whole community of people who take pride in testing. People that take testing as a craft. If you do testing well, it is an original, creative, challenging task. A task that tries to understand the purpose of the product and tries to find ways in which the product can be misused or improved. These people are about as far from “clicking machines” as you can get. Yet they are lumped together in the same bucket. As testers. That do testing. Like, you know, low value clicking and checking.

It took me some time to understand, but at the heart of this discussion of “testing vs. checking” is the esteem of real testing. And the self-esteem of people who take pride in contributing to a high-quality end-product. So if you ever meet someone who is emotional about the distinction of testing vs. checking, that person probably sees himself as a real tester, someone that honors the craft. Pay him respect by making that distinction.

But it is even more: It also is about marketing. If you want to “sell” testing and promote it, you need to make that distinction. (It is no coincidence the term was coined by consultants.) To make an analogy: The situation is that the current notion of testing is a bag full of sand and diamonds. And when people need to “sell” testing (e.g. to management or customers), they have to sell testing as such. Of course nobody pays the full diamond price, if there is a lot of sand in the mix. So if you want to sell the diamonds, it is important to separate them from the sand. By differentiating, you make clear that testing is important and valuable and needs expertise.

Essentially, this is what this distinction is all about. Testing is diamonds — a valuable and important task. Checking on the other hand is a mere mechanical task with low value that you better automate.

Checking is just sand.

This post has also been published at dev.to. If you liked it, I would be thankful if you clapped (as much as you want), and twittered or shared. Thank you!