Who’s Testing Your Unit Tests?
What’s good for the Goose may not be good for the Gander
Writing great software is hard. Software Engineers are human (probably?) so we characteristically make mistakes that lead to bugs that crash our applications. In my opinion, one of the best tools for stemming the tide of human error is the often misunderstood unit test.
Learning to write code that can be easily tested is a major milestone toward learning to write elegant software. This is because writing easily testable code involves myriad beneficial engineering techniques and patterns. For example, dependency injection allows us to easily mock dependencies in our tests, but the pattern also introduces several additional benefits to the system under test as a side effect. Proper separation of concerns not only keeps the test suite simple, it keeps the system under test simple. Adhering to the DRY principle reduces the test coverage area while reducing technical debt in the software.
Unit tests are code. It stands to reason that the same techniques that make our software better can be applied to the tests themselves. Wouldn’t this make our test suites better, therefore making our software even better? I could tell you that the answer is “it depends” but that would be rude. Instead I will argue that the answer is a firm “absolutely not.”
The Point of Unit Tests
Say we’ve written an interesting, complex, and important piece of software called SuperAmazingThing®. We need to ensure that SuperAmazingThing® does exactly what it says on the tin. So, we write unit tests to prove that the interesting, complex, and important stuff that SuperAmazingThing® does is assured. It is the complexity of SuperAmazingThing® that compells us to write these tests.
Complexity
The aforementioned “beneficial engineering techniques and patterns” introduce a fair, yet acceptable amount of complexity to the system. The important corollary is that applying these techniques to the test suite introduces complexity in the test suite.
For example, let’s examine the DRY principle in the context of unit tests with a little story about a software engineer named John Doge who works on the SuperAmazingThing® project.
John has written a lot of unit tests and finds mocking dependencies to be a tedious, redundant exercise. So, instead of generating a mock object in every single test, he decides to write a helper function that generates a mock object and sets up some common expectations on the mock; he then calls that helper function in all of his tests that need the mock. Eventually he discovers that he has re-used the helper function no less than twenty-two times in his test suite. Such elegant. Much fewer repeating. Wow.
As SuperAmazingThing® is changed and refactored under active development, a scenario emerges where the common mock expectations in the helper need to be slightly different. Does John generate a new, standalone mock? Probably not. It’s much easier to just add a little ternary to the helper function. The test suite is now accumulating some business logic (the business of testing!) that will only evolve from here.
Taking the argument full circle: we lowly humans started writing unit tests because our imperfect brains cannot write perfect, bug-free software. If that same imperfect brain adds complexity to the unit tests, we will have engineered a conceptual stack overflow. Are you writing tests that ensure your complex test suite abstractions work as they are intended? I doubt it.
Keep Your Test Suite Simple, if not Verbose
When it comes to unit tests don’t worry about repeating yourself. Every test should uniquely encapsulate the unit under test and all of it’s dependencies. This advice may garner concerns that you will need to refactor a lot of unit tests when something changes. If you find this to be true then you may want to evaluate your test suite to make sure that your unit tests are not secretly integration tests in disguise. If you are certain that this is not the case, maybe the system under test contains too many blocks of similar logic in disparate places — that’s where you should spend time applying the DRY principle instead of in the tests.
I should mention that some testing frameworks provide abstractions that can cut down on repetition, and those are fine to use. For example, PHPUnit provides a pattern called the Data Provider that allows you to run the same test with many different data inputs. Your mileage may vary depending on the language and test suite you’ve chosen, but the point is that these built-in abstractions are safer to use than ad-hoc abstractions because the test suite tool is probably unit tested itself. Testception!
Finally, every time you get the urge to add complexity to your test suite — or better yet, when the tests start becoming inherently complex with lots of mock objects and conditions — it’s time to re-evaluate the system under test. Consider a humble object abstraction: your unit is probably doing too much for a single piece of code, so break the logic out into another class. Then you can test that new class separately and mock it out in the current test.
Have any other advice about keeping unit tests simple? Let’s talk about it in the comments. Thanks!
