Unit testing, you’re doing it wrong

TLDR; Existence of untested code in the wild should worry you: most of our lives is software controlled in a way or another. Good news is that you can do something about it. Bad news you may be doing it wrong.

Disclaimer

I understand that I am addressing a very sensitive topic; I will probably offend many readers that will say that I am an insane troll and my views are bullshit. Offending is not my objective, but I stand by my opinions. Of course comments are here to for you to voice your opinion. And yes this piece is biased by my past experiences, but that’s the value of it, sharing my experiences.

‘How legitimate are you?’’

Fair question. I have a 35 years long career in IT; I have worked at companies of various sizes and culture. I have often been in some transversal position and had the opportunity to meet and work with a lot of developers (think thousands). While most of my roles involved code, I also touched on QA and BA activities. I am now in CTO-like positions for 2500 ITs and had the great privilege to work with well-known french experts, as well as lesser-known ones.

Some fallacies about unit testing

1. TDD is all about unit tests

Unit tests, really?
Unit tests

2. Automated testing is all about unit tests

No, automated testing describes a process: having tests automatically run as part of your build/delivery chain. It covers every kind of tests you can perform automatically: behavior tests, stress tests, performance tests, integration tests, system tests, UI tests….

Having a behavior test would help

3. 100% code coverage requires extensive unit testing

NO, NO, NO and f…g no. In a perfect TDD world, untested code does not exist in the first place.

Yep, let just add another stupid test

4. You have to make private methods public to reach 100% coverage

I need to assess the private parts

5. Some code does not need be tested

The design of the Death Star is Rebel proof, right ?!

6. You need to use a mocking framework

Look what I have built !

7. Tests are expensive to write

Yes, testing is expensive in most of industries: think about testing a home appliance, a drug or a new car…

This test is expensive!

8. The ‘testing pyramid’ is the ultimate testing strategy

You probably have heard of the testing pyramid. Basically, it describes that you should have a lot of unit tests, less component tests, then less integration tests, and so on, up to the top of the pyramid where you have a few use case based/acceptance tests. It is used as the default testing strategy for most projects.

Pyramids can be dangerous!
  • Having a lot of those means you find issues as soon as they are introduced in the code base, but significant re-designs of your solution will be ridden with failing tests.

What about some truths ?

1. Unit tests are not about testing a method in isolation

Here is what Wikipedia proposes:

Isolation

2. 100% coverage does not mean your code is bug free

This the first rebuttal I get whenever I talk about 100% coverage. Of course, it does not.

This is deep

3. There is a tooling problem

The truth is unit tests are in the spotlight mostly thanks to tooling! We should be all eternally grateful to Kent Beck for creating sUnit, the library which triggered a testing revolution, but we must not stop there.

4. It is difficult

Yes, but this is no more difficult than designing the software up front. You face complexity, but what is interesting in test first approaches, is that you have an opportunity to focus on essential complexity as test code ought to be simpler than actual implementation.

Tests are hard to write
  1. it constrains your design, and nobody likes extra constraints
  2. it gives you the impression of being unproductive
  1. Constraints help you drive the design. And as you are problem first, this is bound to be a good design.
  2. Worst case, tests will be thrown away. But they helped you build a solution and a deep understanding of the problem. At best, they prevent future regression, and provide help and documentation for future developers.

5. Tests require maintenance

Maintenance is an ongoing effort

6. Having too many tests is a problem

Since tests need to evolve with the production code, too much tests will hamper your productivity: if changing some lines of code break hundred tests or more, the cost (of change) becomes an issue.

Too much tests…

7. Throwing away tests is a hygienic move

It should be obvious by now that you need to keep a manageable number of tests. Therefore you must have some form of optimization strategy for you test base. Articles are pretty much non existent for this kind of activity, so let me make a proposal:

  • make regular code coverage review, identify highly tested lines and remove tests you find redundant.
Recycling

8. Automated tests are useful

Last but not least. Automated tests have a lot of value. Yes, a green test looks useless, like any security device: safety belt, life vest, emergency brakes…

Incorrect error handling

Conclusion

First of all, thanks to you for having the patience of reading this so far. I’d also like to thanks my reviewers for their feedback, namely: Alexandre Victoor, @MorganLeroi, Thomas Pierrain and yannick grenzinger!

  1. Test behaviours, not implementation. A.k.a. understand what unit stands for in unit tests.
  2. Maintain your test base with the delicate but strong hand of the gardener: gently refactoring when necessary and pruning out when no longer useful.

Cyrille Dupuydauby. Convinced crafter , NewCrafts and Devoxx speaker, OSS maintainer, coding architect. Founder of the Speaker Academy

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store