Test cases are a great tool. But not for finding bugs.

Just out of curiosity, we did two test rounds with a team of consultants abroad. We split the group in two and gave different assignments to both of them.

Photo by Helloquence on Unsplash

On the first assignment, the team was given a very clearly written test specification based on the product’s requirements. We counted 18 bugs as an outcome of running 120 short test cases.

The second assignment we gave was only a short briefing. The mission was to test this product to your best knowledge and report all issues in a way you find best suited. We don’t want any documents, only Jira tickets.

As a result, we received 25 bug reports. In addition to that, the testing team (or the second half of it) got creative and used video capturing to demonstrate the bugs. The software developers also gave thumbs up!

For the time spent in executing testing work, the result received was 39% better when the testers put their focus on finding bugs and not following the test cases.

Yes. I know it was just a simple experiment with lots of variables that we didn’t take into account. But it was fun to play around with possibilities.

Of course, this is common sense. Obvious to most testers. We find more bugs by exploring than by repeating those recipes we call test cases.

Test cases are a great tool when we need to create evidence to back up our contracts or legal requirements for example. But then we should first accept that the goal is to create documentation, not to improve our product.

If on the other hand, we want to build better quality software, finding the bugs is a must.

So go out. Explore. Fool around. Play some. Have fun and keep an eye out for bugs on your path.