How to Create Robust and Useful Unit Tests
In my previous company I worked on a major enhancement to a legacy application where we inherited a smorgasbord of unit tests — approximately 7,000 tests in total. Whilst enhancing the application we spent an inordinate amount of time fixing broken tests. Of course requirements changed and so it was necessary to re-write tests to reflect this. But in this case rather than being facilitators, the tests became an impediment to innovation and change, and hence an impediment to quality. The unit tests were fragile.
So how could this have been avoided? In this article I’m going to be outlining guidance on how to structure well factored and purposeful unit tests. But before I do, a quick note about purpose.
The primary purpose of a unit test is to verify that a unit of code functions as intended, with the actual results of executing the unit of code matching the expected results.
A secondary purpose is that the process of engineering unit tests leads to well factored code.
And a side-effect of unit testing is to provide documentation for components in the system. A set of unit tests reveals the contract and intended behaviour of a component, and as such is a great starting point for an engineer when charged with enhancing a component. Well factored tests with clear intent enable new developers to use the tests as a guide to the functionality that the system under test encapsulates.
1. Concerned with testing only one unit of code
Each test class should target only one system under test (SUT). And each test method should target only one SUT method. For example, when implementing MVC a controller may be your SUT, and an action handler on the controller your SUT method.
As it says on the tin, it’s about testing a unit of code. To achieve this it’s necessary to isolate this unit of code from its dependencies. A lot of modern languages enable this through providing IoC capabilities such as dependency injection.
If it’s not possible to isolate a unit of code for testing purposes then you might want to consider refactoring the code.
2. Concerned with testing only one behaviour
Some purists take this to the extreme and limit each test to only one assertion.
But typically I find that a behaviour equates to a path in the code. So as a starting point aim to have one unit test per path, with as many assertions as required for that path.
However, sometimes paths overlap. For example, where a method has guard clauses before branching. The guard clauses are behaviours in their own right, and so are candidates for their own tests.
In the aforementioned project one of the engineers changed the behaviour of a guard clause and then had to fix 12 failed tests, as they included duplicated assertions for the guard clause. Do not repeat the same assertion in multiple tests.
3. Independent
Each unit test should be able to run independently from the other unit tests.
Tests should be stateless with no dependency on other tests, and no intention of transferring state between tests. Where a test requires the setting up of test fixtures prior to acting then this should be done in the test itself.
Tests should be able to run in any order and so a test should not assume that a previous test has been run. Many test runners are in fact multi-threaded and so do not guarantee the order in which tests run.
If multiple tests require the same set-up code then refactor this code into a local method. Name the method appropriately and explicitly call the method in the arrange phase of each test, so that the intent is clear.
4. Intent should be clear
The intent of a test should be clear - so that an engineer can easily see how the system under test is supposed to behave. When asked to modify the system under test, or diagnose why a test is failing, the engineer will be able to carry out the task more easily and with more confidence.
A test should do what it says on the box. So adopt a naming convention that incorporates the following -
- Method name
- State under test,
- Expected outcome.
My preference is [MethodName]_[ExpectedOutcome]_[StateUnderTest]. For example,
Create_ThrowsDuplicateNameException_WhenAnotherProductHasTheSameNameYes, it’s a long name, but the intent is clear.
You may have a different preference, but that’s fine. The important thing is to describe intent and be consistent.
In addition to having a good name, you should code your test with the goal of revealing intent. Do not use magic strings, and explicitly name your local variables and helper methods.
5. Deterministic and repeatable
The aforementioned project made use of a C# testing library called AutoFixture to take the heavy lifting out of creating the test fixtures. It’s a very useful library which speeds up the development of tests and reduces the number of lines of code. But many of the default type builders generate instances in a non-deterministic manner. If the outcome of the test is dependent on these instances then the outcome may vary each time the test in executed. A test that passes one time, may fail the next.
I saw this happen, with a test passing over many executions, and then failing out of the blue with no hint as to why it failed as the test and test subject code was not recently modified. In this case the test contained a mathematical calculation that eventually failed due to a rounding inconsistency.
To be fair, Mark Seemann, the author of AutoFixture, is very clear that it should only be used to generate anonymous fixtures, which by definition do not ‘determine’ the outcome of a test.
6. Avoid logic in unit tests
Unless you want to write unit tests for your unit tests, don’t add calculations or logic to your unit tests. The code needs to be simple, with a cyclomatic complexity of one.
Logic obscures the purpose of the tests, and even with the purpose explained, the complexity of the code may reduce the readers confidence in the test.
Conclusion
I’d like to conclude by saying that unit test code should be treated as a 1st class citizen rather than an unwanted afterthought.
I’ve seen engineers writing high quality well factored classes, and then cut corners when it comes to writing tests. They may achieve 100% test coverage but the test code is not well factored and so is difficult to read and is fragile.
Take pride in unit testing. I personally get as much enjoyment from writing a well factored set of tests as I do from writing the test subjects. Once you embrace unit tests as 1st class citizens then writing unit tests becomes very satisfying.
