Using Coverage to Improve Your Unit Tests
Measuring the coverage of your tests is easier than ever. In a modern IDE the button to run your tests with coverage is right next to the ones to run or debug them. The coverage results are presented class by class with little chart graphics, as well as relevant lines being highlighted in colour in the source code.
Coverage data is easy to get hold of. What is the best way to use it though?
When you’re writing new code
Most people agree you should deliver unit tests together with all the code you write. You can argue about which order to do things in, but in my experience what works best is short feedback loops. Write a little test code, write a little production code, and build up the functionality together with the tests. When I’m working like this I will run the tests with coverage from time to time as an additional insurance that I haven’t forgotten to test some new code I’ve just written.
The main danger here is that you become very satisfied with a high coverage number and don’t notice you’re missing both code and tests for a crucial piece of functionality. Perhaps you forgot to add error handling. Perhaps you missed out a business rule. If you never wrote the production code in the first place, then coverage measurements can’t discover that for you.
When you have to change code you didn’t write
Modifying code that you didn’t write yourself, that has poor or missing tests, can be challenging — particularly if you don’t really understand what it does, but you still have to change it. When I’m faced with this situation, test coverage is one of the ways I learn about how good the tests are, and which parts I can refactor more confidently.
I can also lean on the coverage data to discover new test cases and increase the covered areas. This can get dangerous, though. If I write tests purely to increase coverage, I can end up coupling the tests quite closely to the implementation.
When you’re working in a team
One of the characteristics of a team is that you have ‘norms’ or accepted behaviours that everyone agrees on, whether implicitly or explicitly. One of your team norms could be that you make coverage measurements part of your code and test review process. It can help you to see where tests are missing — perhaps some team members need more support and training to write better tests. It can also be encouraging when you see complicated new functionality is well covered.
If you regularly measure test coverage for your whole codebase, I would encourage you to look at trends more than absolute numbers. I’ve seen arbitrary coverage targets lead to people preferring to test only what’s easy to test. People can avoid doing refactoring because it will introduce new lines of code and lower their coverage overall. I’ve seen tests written with missing or very weak assertions just to improve the coverage numbers.
Coverage is supposed to help you to improve your unit tests, and unit tests are supposed to make it easier to refactor. Coverage measurements are a tool to help you to improve your unit tests and make your life easier.
