Test Driven Development Wars: Detroit vs London, Classicist vs Mockist
I remember when I first started learning to program, and one of my teachers explained the world of software development to me in a single sentence:
“It’s just a bunch of people in a room arguing with each other”
He went on to describe how you’ll never find a consensus agreement on some core issues like testing and design. One senior developer might preach one view, and another will preach an opposing view. One will insist you write a feature spec first, another will deny your existence until you start at the unit level.
In hindsight, it would have been useful to pay more attention to the opposing tribes of testing and their various disagreements. Navigating through the web and finding numerous articles that flat out contradict each other can be mind opening but admittedly confusing. Which practice do I apply in my day to day? What if the Classicist view pleases me and makes intuitive sense but infuriates those in my team?
But what is Classicist and what is Mockist?
These two camps are commonly referred to as the Detroit School of TDD (or Classicist, state based testing, Inside-out, black-box testing) and the London School of TDD (or Mockist, Interaction testing, Outside-in testing, white-box testing). Both agree that TDD is an effective design tool, but how they apply TDD differs enough to classify them into two schools of thought. For the duration of this article I will be using these terms interchangeably.
You may have heard of famous Classicist TDD practitioners including Uncle Bob and Kent Beck. If you want an example of a Mockist look no further than Sandi Metz, J.B Rainsberger or Steve Freeman
The Detroit school of TDD starts with an inside-out approach to testing. Testing begins at the unit level, and the design is supposed to “emerge” from the tests. I’ll admit that whenever I read this I cringe a little. Just because you’re writing tests doesn’t mean your design will blossom into an enviable work of technical art. To write a good design you have to learn about, well, design.
You can learn all you want about testing strategies and theories, but without some grounding in design you can’t expect to build a well defined and extendable system that’s going to be immune to future business developments.
A key feature of the Detroit school is the avoidance of mocks. It was only upon reading this that I realised I had been taught for years only the London school of TDD (I do live in London, after all). I always assumed this was The Way. Say you have a ledger class that performs some business logic, and at the end has a calculate method that delegates to another object:
def calculate_total_balance(x,y) # complex business logic that mutates x and y some_calculator_object.sum(x,y)
end
The Classicists would test this by making an assertion on the calculate_total_balance method and assuming both x and y are 2 and 5, the final test would expect a value of 7. So the Detroit school is not afraid of testing the collaborators of an instance indirectly through the interface of another class. No mocks would be introduced here.
If a practitioner of the London school were testing this individual method, the expectation would not be 7, but we would have something like
expect(any_instance_of(Calculator)).to receive(:sum).with(2,5)
The Mockists argument is that the sum method of the Calculator instance will be tested elsewhere, and we would not want a unit test for our Ledger class to break because of a change to the implementation of Calculator. Testing that the message gets sent between the objects is sufficient enough. Whilst we haven’t confirmed through this unit test that the behaviour actually works, we’re confident that it does because we’ve tested the interaction between the collaborators and that there will be another unit test that actually makes assertions on the Calculator sum instance method.
The Classicists will insist that the Mockist approach does nothing to confirm the working state of the application, and also ties the tests to the implementation. If we change our method above to use, say, a CalculatorService, then the behaviour would remain the same and we would still have the number 7 returned, however our test would now produce a false alarm despite everything working fine.
The Mockists come back and say that because our unit tests should be isolated, we wouldn’t want a breakage in one collaborator (say, the Calculator) to lead to failing tests amongst their callers. Yes the application is broken, but we would prefer a single test to fail instead of multiple tests. Also that’s what integrated tests are for; to test the integration between multiple objects
Another way to describe the differences between the Classicist and Mockist view of testing is the use of state based tests and interaction based tests. So as you can see from the Mockist example, the London school would emphasise testing the interaction between objects and ensuring the correct messages get sent. The Detroit school argues that having a more state based testing architecture would afford ruthless refactoring, and with mocking excessively you are tying your objects to their collaborators interfaces and therefore creating a more inflexible test suite.
Mocks break the old mantra that one should test the interface, not the implementation. Changing the nature of the calls to the collaborators in any way would break the tests regardless of whether the functionality was in a working state. This is one of the common criticisms of the London school of testing.
You’ll often hear about black box testing vs white box testing. Black box testing is associated with the Detroit school, whilst the latter is associated with the London school. So just like our ledger example above, the Classicists would not mock the collaborator object of our ledger class but instead treat the entire method call like a black box. I call a method, and I expect this result to come back. I don’t care how.
The Mockists would prefer white box testing by saying: I call a method, and I don’t care about the general correctness of the method because this is communicating to another object that I will test the return value of elsewhere. All I care about is that it communicates effectively.
I wish I was smart enough to end this blog post by giving a solid recommendation on which testing approach is better. However like a lot of things in software, there are tradeoffs.
If we were to classify useful, highly valuable tests that can deliver real business value, I would make the case that effective tests:
- Have a high chance of catching regressions
- Have a low chance of producing false positives
- Provide quick feedback
It’s often very difficult to maximise every single one of these without sacrificing one of the others. Whilst pretty much all unit tests provide quick feedback, and both Mockist and Classicist test setups provide a nice safety net to catch regressions, it’s the second point which is often contentious. A test architecture designed by a London school practitioner, with an extensive set of mocks, will likely produce many false positives and handicap any future refactoring efforts. Whilst a Classicist test suite will include less isolated unit tests that would break due to no fault of the particular subject under test. Redundant Coverage is an anti-pattern where a piece of production code is depended on by multiple tests, and it’s something commonly associated with the Detroit school.
I had a fun time researching the different trains of thought amongst these opposing camps. Since learning to program I have mainly been exposed to Mockist inspired testing suites and London school practitioners. I’ve had people tell me it’s diabolical to indirectly test the return value of a collaborator through the avoidance of mocks. Having knowledge on the opposing opinions and trade offs between each approach is invaluable and ensures that when I see a code base with no mocks, I shouldn’t be so quick to judge. And neither should you.