iOS Unit Tests and Asynchronous Calls

How to make tests fast and reliable by using virtual time

Time matters

For code compilation and test execution speed is critical. The faster you receive feedback as a developer, the better.

With a growing team an interesting question emerges: How many commits per hour does your CI system allow? In early 2021 that number for us was 4. And the reason? Before merging to the main branch all unit tests must run successfully on a feature branch. The feature branch also needs to be up to date, containing all the commits from the main branch. Compilation and execution time was nearly 15 minutes.

At Mobimeo we have about a dozen iOS developers working on the same git repository. A capacity of only four commits per hour was beginning to affect our productivity. So we started to work on improving our build and test execution times.

• The first step was to remove the virtualization of our Macs. It turned out that using VMware Fusion was doubling our execution times.
• The second step was to not compile our third party code every time, but instead to use precompiled binaries. This step is worth a dedicated article that might come later.
• The third step was dealing with asynchronous calls in our test suite. Improvements here shaved a full minute off the execution time for the entire suite. All tests combined now only take a couple of seconds to run.

This article describes how we preserved our asynchronous code but made it synchronous in our unit tests. Most of our refactored code used ReactiveSwift. In this article we use Combine for examples, but the concept (and sometimes even the code) is also valid for RxSwift and ReactiveSwift. We start with a simple class using DispatchQueue.main.asyncAfter(), so even if you don’t use any reactive framework, this article might still be useful in helping you improve your unit tests. Code examples are written using Swift 5.5 and Xcode 13.2.1.

The problem with asynchronous calls

Let’s assume we have a simple class using DispatchQueue to defer some action:

This class stores a string, but does so with a delay of one second. A unit test could now assert that after 0.5 seconds the string has not been stored, but after more than 1.0 seconds it will have:

The problem with this code is now that it will take at least 1.2 seconds to execute, no matter how fast the hardware is. We could make the delay configurable and set it to a smaller value during testing, but in more complex scenarios this time might be computed and hard to change. So we might want to keep the 1 second delay. Also depending on short delays like this will likely make our tests flaky. If the test usually takes 0.1 seconds to execute and we wait for 0.2 seconds, then sometimes the machine takes a bit longer and we have a false positive.

Making DispatchQueue.main.asyncAfter synchronous — in unit tests only

We are now going to change the class Delayer, so that it keeps it’s functionality in production, but is synchronous in unit tests.

Since iOS 13 and macOS 10.15 DispatchQueue implements the Combine protocol Scheduler. In this case schedule(after: ) does exactly the same job as asyncAfter(deadline: ) in the earlier version. We now use Dependency Injection to use a TestScheduler in unit tests.

Previously we used DispatchWorkItem.DispatchTime.now(), which is always the current system time. Instead we use scheduler.now, which in production is the system time. For unit tests however this will be a virtual time that we control. More details about that soon, but first the whole Delayer.swift after the refactoring:

As you can see, if this class is initialized without arguments as before, it will use DispatchQueue.main and keep the same behavior.

The eraseToAnyScheduler function is part of the open source library CombineSchedulers by pointfreeco. This is necessary to conform to the protocol AnyScheduler<DispatchQueue>. We can’t simply write var scheduler: Scheduler because Scheduler has associated types (SchedulerOptions and SchedulerTimeType) and the compiler would complain. CombineSchedulers helps us here for convenience.

Using TestScheduler and virtual time in unit tests

CombineSchedulers also contains a class TestScheduler which conforms to the protocol Scheduler. In contrast to the schedulers which are part of Combine, it uses a virtual time internally. This is a simple property, which is not affected by the system time. We tell it when and how time progresses. Let’s see what this looks like in our unit test:

The new testCombineDelayer uses advance(by: ) to control the virtual time variable and increase it. The virtual time is an internal property that is increased by this. All scheduled actions due in this timeframe are executed. This happens synchronously, so no there is no longer a need to wait for XCTestExpectation. Note: The original test code still works. It’s left in for reference, it can be removed.

Result: fast, stable, easy

The resulting unit test has several advantages:

• Speed: While this one unit test took about 1.2 seconds before, now it takes only 4 milliseconds.
• Flakiness: Previously we experienced sometimes randomly failing tests, they were flaky. After our improvements they are stable.
• Debugging: It’s now possible to set a breakpoint and pause a test during the execution. As execution occurs on a single thread, this does not change the behavior anymore.

Unit tests by definition must be fast, isolated and repeatable. By using a virtual time our asynchronous code is now also testable following this definition. And we succeeded having significantly faster build/test times on our CI machines.

Links: TestScheduler in different Reactive Frameworks

The concept of using your own virtual time in unit tests is widespread. All main reactive frameworks for iOS have a class called TestScheduler supporting this; for Combine there are third party libraries available:

• Combine, by pointfreeco: documentation, sources, video tutorials
• Combine, part of Entwine: documentation, sources
• RxSwift: documentation, sources, raywenderlich tutorial,
• ReactiveSwift: sources

A repository with different iterations of the Delayer, unit tests and an implementation using ReactiveSwift can be found here: https://github.com/carsten-wenderdel/TestSchedulerDemo

Thanks for reading! And a big thanks to my colleagues Dima Hutsuliak, Matthew Bryant and Mohamed Fouad for proofreading and giving valuable advice!