My Big 5 takeaways from iOS Unit Testing by Example

I recently worked through the entire iOS Unit Testing by Example: XCTest Tips and Techniques book by Jon Reid (2019).

The book itself is full of great tips and techniques to help us learn how to better test our iOS applications. In addition to familiar techniques of using dependency injection, spies and mocks; the book also teaches us how to leverage unit tests to also cover view controllers and simulate interactions that can potentially reduce the need for certain UITests whilst improving our test coverage and increasing test speed.

To keep this a sensibly sized post I will restrict my reflections to these 5 concepts that were the biggest takeaways for me:

• Test code should be cared for as much as production code
• Code coverage is a useful metric but doesn’t tell you everything
• Tests should focus on testing the behaviour of the system not the implementation
• Tests should be fast and run on the appropriate test scheme or test plan
• Good tests support disciplined refactoring

1. Test code should be cared for as much as production code

I’ve worked on legacy codebases that often have test files with lots and lots of repeated code that follows the same pattern. In addition to violating the DRY (don’t repeat yourself) principle, they sometimes don’t actually capture the essence of what we wanted to test, making it hard for others or our future selves to reason about what exactly it was we were testing.

”Test code is just as important as production code”
— CleanCode: A Handbook of Agile Software Craftsmanship

Taking the time to refactor tests to reduce repeated setup code, using good naming for any properties that are used and extracting helper functions with clear naming for both factory methods and assertions can be a great way to make our tests readable and express precisely what we are testing and the expected result.

Anytime we see duplication in test code we should feel the urge to refactor much like we do in production code.

In the following example we’ll go through such a file:

0. Starting point

This simplified example illustrates some common issues that we may come across on project test suites where setup code and assertions are copied across without thought and adjusted but without refactoring after.

Duplicate code and assertions without clear context can become hard to decipher and with lots of duplication that is not needed. However, with just two cases there can be an argument that it is acceptable for now.

1. The Problem

However, when we start to add more screens to this example we need to update every test to ensure we are covered. This compounds with each addition and increases the risk that we missing some places and will gradually makes the code less and less readable / comprehensible.

So this is an example where we should consider refactoring to reduce the duplication and to protect the individual tests from changes and the need to update in multiple places.

2. Lift setup code
A simple first step is to lift the creation of the SUT (system under test) to the setup test fixtures reducing the duplication of boiler plate code in each test.

3. Use helpers

On extracting helper assertion functions for readability and to reduce duplicate code we get a far more succinct test which clearly shows the three A’s of testing (Arrange, Act, Assert) or, if preferred Given, When, Then of testing:

An additional benefit is that each test is protected from changes and if any do need to be made they are centralised in the helper functions.

4. Adding another screen

Now, adding a new screen to tests is far easier, we just need to add the appropriate assertions to our helpers and the actual test for it:

Adding a new screen only requires updates to the helper functions.

These changes, in my opinion, make for a far more readable and robust test suite. The full file is now as below.

Note: in the full file I added filePath and line to the helpers which enables Xcode to report failed assertions from where they were called rather than within the helper itself which is a really handy feature of XCTest.

Full file is now more readable and robust.

So my key takeaway was not to just lazily duplicate tests to cover the cases. Take the time to care for your test code as much as you care about your production code to write clear expressive and readable tests. Refactor our tests and craft them as well as we would our production code.

2. Code Coverage is a useful metric but doesn’t tell you everything

Code coverage only measures whether each section is traversed during tests. It doesn’t check whether the tests are actually testing anything on those lines. So in the most extreme a 100% covered module could be completely untested.

In the following contrived example we have a class with a single function that returns a Bool if the number provided is odd or even.

In the tests we have deliberately missed the assertion of the value returned:

Since all branches of the isOdd(_:) function are traversed we still get 100% coverage despite we haven’t actually tested anything!

Code Coverage reports 100%

The example above is a very simple function to test and reason about, in more complicated code under test we may accidentally miss some important cases and leave them untested with the code coverage report leading us to believe everything was tested.

Test coverage, then, while being a very useful metric that can help us measure progress over time, cannot be overly relied upon to determine the quality of our test coverage.

3. Tests should focus on testing the behaviour of the system not the implementation

An issue I have come across working on some legacy codebases is that when a change is made to the production code implementation, corresponding tests need to be updated to match them in lots and lots of places. It is better if we can focus our tests on testing the behaviour of the component in question, rather than the implementation.

This change of perspective can us make our tests more robust to change and therefore allow us to make changes more readily in production code whilst still maintaining the existing behaviour.

Combined with the refactoring of test code extracting helper methods mentioned above, we can effectively protect our tests from changes in the code that do not concern them.

When something does need to be updated in the test code it can be done in the extracted helper methods/setup in one place (e.g. if the SUT needs a new init argument we only need to make the change in one place). This way, despite the change, most of the test file remains unchanged as the tests only test the behaviour and are not concerned with what happens ‘under the hood’.

4. Tests should be fast and run on the appropriate test scheme or test plan

Since the publication of the book, recent versions of Xcode give us access to TestPlans, which can be used in a similar way as Jon uses test schemes in the book.

We want our tests to be fast to support our development process. To foster this it is very useful to ensure that they are running on the appropriate scheme.

For example when implementing a new feature with lots of frequent changes it is not a good idea to always be running all test schemes at once (looking at you UI Tests). As this will greatly slow down the feedback loop that good tests allow us to enjoy.

Following on from this snapshot tests too should be run on a separate scheme dedicated to them as they do take a little longer to run than pure Unit Tests. When we make changes to the UI then we can run these tests and update the reference images as needed.

Expanding on these examples that Jon Reid discusses in the book, something I learned from the excellent iOS Lead Essentials course I am studying. For pure business logic that is platform agnostic we can actually run these tests on a MacOS target rather than a simulator. This will greatly speed up the Unit Tests on those areas of the code.

The benefit of breaking the module into these distinct test schemes is we will greatly increase the speed of the tests that need to be run for the changes we are making on a particular area of the codebase. While this may sound like a little bit of overhead the ability to run tests in a few seconds helps to build a really nice safety net whilst we are working.

Of course all tests should be run on a combined scheme for CI and locally prior to pushing code to ensure that our changes didn’t break anything.

5. Good tests support disciplined refactoring

The disciplined refactoring section was a real hidden gem and unexpected given the title of the book. In many ways this will be the section I lean on the most going forward in my own projects.

When refactoring having fast reliable tests that cover all the behaviour of the code that we will be changing is a crucial starting point. We can refactor step by step and run tests as we go, ensuring that we don’t make a breaking change along the way. If we do break something we can easily go back one step and correct the issue that caused the test failure and then continue.

Although we are going slowly one tiny step at a time, we will end up being faster as we won’t have a ton of cleanup to do after a big refactor change that broke a lot of tests that we were unaware of as we waited to run them until the end of the refactoring.

However, for this approach to be truly effective we want our tests to be fast and provide good behavioural coverage of the code we want to refactor. If we are working on legacy code then we should add tests as required prior to refactoring to ensure we aren’t inadvertently making breaking changes.

Once we are happy we have brought the component under test coverage and have it’s behaviours well covered we can commence with the disciplined refactoring.

Disciplined refactoring is moving in small steps, verifying each step as you go. Unit tests are the secret sauce that make such refactoring possible. — Jon Reid, iOS Unit Testing by Example (2019)

The book then outlines some step by step methods for a variety of refactoring tasks. After each step we run tests to ensure tests all pass before continuing the refactor. Should something go wrong we go back a step, fix the thing that caused the test failure, test and then continue.

Final thoughts

Here’s a quick recap of my 5 biggest takeaways from the book.

• Test code should be cared for as much as production code
• Code coverage is a useful metric but doesn’t tell you everything
• Tests should focus on testing the behaviour of the system not the implementation
• Tests should be fast and run on the appropriate test scheme
• Good tests support disciplined refactoring

Tests are often an afterthought but improving our ability and approach to them can help us to unblock our code. Creating malleable code that can be easily refactored with confidence, whilst maintaining the integrity of the system, requires a foundation of FAST accurate tests.

I would definitely recommend you take the time to work through the book yourself if you are interested in improving your testing skills or curious about other approaches.

Jon Reid’s book can be found at The Pragmatic Bookshelf

Special thanks to Marco Guerrieri, Raynelle Alphonso, Pete Murray, Trevor Doodes for reviewing the draft of this post.