Mocking is not practical — Use fakes
This article talks about the benefits fakes provide over mocks primarily in areas where fakes and mocks can be used interchangeably (ie. everywhere in theory). However, if you’re dealing with a badly designed API, mocking might be easier and/or better.
Code sample here.
What are mocks and fakes
Definitions are already exclusively covered in many articles online. I’ll just mention them here for completeness.
Mocks are objects having the same interface as real types but can be scripted to return preprogrammed responses. They register calls they receive which can be verified as part of test assertions.
Fakes are objects that have a fully working implementation as the real type but optimized and simplified for testing purposes.
What makes fakes better?
Black box testing/State verification
The difference between testing with mocks vs fakes is that mocks are used to test behavior whereas fakes are used to test state.
White-box testing: The process of mocking calls made by the SUT (System Under Test) on its dependencies using when statements and then verifying implies the test knows and dictates exactly ‘how’ the SUT should behave. This is white-box testing.
Problem with white-box testing: Any change to the implementation like algorithm optimization, code cleanup and/or changing the order of statements can cause the test to fail. These changes lead to updating the test fixture making the test ‘follow’ the real implementation. Such a test is nothing but a duplication of the production code and needs to be maintained like it.
Black-box testing: Testing state with fakes is a way of writing consumer-oriented tests. The SUT is considered as a ‘black box’ and assertions are only made against the output of the SUT. Any changes to the real implementation are valid if the tests confirm so. Such tests are robust. They specify the ‘what’ and don’t care about the ‘how’ (which is what an API consumer would do anyways). This is black-box testing. (Note that SUT need not be a single class)
Black box testing makes us think of the API as a consumer. If it’s not easy for you to assert state in your tests, it will not be easy for your API consumer to use your API.
Favor tests as safety nets
Tests should be guard rails for refactoring. If algorithmic refactorings force you to update tests, you are in effect modifying these rails themselves. It’s like molding the mold. Not only do you end up changing specifications as you like, but its more work. This defeats the purpose of tests.
Favor tests as API documentation
A consumer-oriented test acts as documentation. The test name is a summary of what the SUT does in given conditions and the test itself explains the API capabilities, usage, and edge scenarios. This establishes a clear difference of purpose between test code and production code.
For behavior verification tests (with mocks), the test describes ‘how’ the SUT interacts with its dependency (not something a consumer would care about). This is also documentation but redundant (We can just read the production code for this).
Favor functional/integration tests
Fakes created for unit testing can be readily used to write functional/integration tests. This is difficult with mocking.
Favor usage of real implementations
You don’t need to fake everything. If you have a pure function or a pure functional type, it is completely controlled by its inputs and monitored by its outputs. It can be used directly in tests. In contrast, the mock approach is to mock all dependencies regardless.
Another Example: Consider a SUT that is not a class but a family of classes. For example, if a class is injected with abstract factories, you can use the real factory and only fake the products it produces.
Optionally, using DI frameworks, to create such test fixtures with real instances and fake products is even easier, cleaner and reusable.
Favor test verbosity reduction
Mocking tends to ignore logic not directly exercised by the test. Such logic, however, would have executed in a production scenario. Strict mocking avoids this but now tends to make the test fixture verbose.
Since fakes are fully functional implementations, all logic is exercised at all times (like production). The test verbosity is lower and relevant to the test.
Richer than mocks
Fakes can be made richer in terms of functionality than mocks. For example, a fake can provide logging capabilities. So unit tests can generate logs if required thereby making it easier to debug.
Favor Single Responsibility Principle (SRP)
Writing a when statement when mocking makes us really focus on the exact method being mocked at that point. We don’t really care how the dependent class we’re mocking works as a whole or how it’s structured. Heck, we wouldn’t have cared if that method belonged to a different class! This can lead to the class API being badly structured and break SRP at the class level.
Even at the method level, practices of using an ArgumentMatcher like any() tend to disregard the method signature (we wouldn’t have cared if the method had few more parameters because we’d just use another any()!). In the test, we only care what the when statement returns.
If a method does two things, mocking it is easy but faking it is harder. Similarly, a type doing multiple things causes it’s fake to also mimic multiple things thereby making it hard to build and maintain. So why not make them lean and just do one thing?
Tests should help in API design. It should be easy to decouple and decompose classes and components by refactoring where the tests confirm that the new design works as expected.
Example: Functional Use Cases is a pattern for creating business logic units that obey SRP.
The price of fakes
Verification API
If you want verification mechanisms like a mock, you can add extra methods to your fake that track this (eg: FakeUserRepository.getUserCount()). This is added effort but keeps you away extensive use of libraries like mockito which can be easily misused resulting in white box testing.
Interface with a single production implementation
It might feel awkward to have an interface for every testable type because you need to create a fake implementation. IMHO, this price is minuscule compared to the benefits this style provides.
Testing the fake implementation
Since a fake is a fully functional implementation, it might become complex enough to be tested itself. Usually, this is an API design code smell and hints on making types leaner so fake implementations can stay simple.
Closing thoughts
Usage of mocks should be limited to situations where creating and maintaining a fake outweighs the benefits described above. Typically, this situation is encountered when dealing with a badly designed API (Could be your API or the library/frameworks that you use).
Though not part of this article, there are ways to abstract such API so it can be faked. Typical abstractions include wrapping/composing in an easily fakable type.
Code example
Most of the snippets shown above are taken from the code sample here.
Further reading
Thanks to E Severin Rudie and swastik sen for proofreading this and providing their feedback. Happy coding :)
