Introducing Mockingbird, a Swift framework by Bird that simplifies unit testing on iOS.
At Bird, we’re committed to replacing car trips with micro-mobility, and that means we need to provide our riders with the best mobile experience possible.
Automated testing is an important part of our engineering process. It’s how we ensure the reliability of our app for all our riders. Mockingbird is our latest in-house testing framework for iOS that utilizes automated code generation to dramatically reduce boilerplate and increase developer efficiency.
The problem: Hand-written mocks are brittle
Even though we benefit from Swift’s type safety when developing our iOS app, the language’s limited reflection capabilities and strong type guarantees make it impossible to have frameworks like OCMock. As a result, we started writing mocks by hand for each test case (let’s call them “artisanal mocks”).
As Bird scaled, we found that these artisanal mocks existed in a kind of “twilight zone” between production code and test code, with few enforced standards or best practices. Even worse, these mocks often made refactoring difficult by coupling tests to production code.
As we hired more and more iOS engineers, we needed a new approach to mocking and stubbing, one that could scale as quickly as our team was.
Searching for a solution
It only took a few days to create a working proof of concept, but we ran into major limitations with Sourcery’s dependency handling. Sourcery only accepts a list of input source files and cannot differentiate between dependency sources and primary sources. Generating mocks for a module would include all of its dependencies, which we would then have to strip out.
Although we liked Sourcery’s robust features, it was too slow to run as part of our builds. Parsing and generating mocks for our project took over 15 minutes.
We also tried existing mocking frameworks like Cuckoo, but we couldn’t find one that ran quickly and could reliably handle more advanced Swift features like generics.
None of the solutions we tested were a good fit, so we built Mockingbird.
Mockingbird takes flight
Mockingbird parses source files and generates mocked types from the declarations it finds. The main advantage of Mockingbird comes from its embedded domain-specific language (eDSL), which provides convenient APIs for mocking, stubbing, and verification.
Mockingbird follows three design principles:
- Easy to install
- Fast and robust code generation
- Simple but expressive DSL
Easy to install
The Mockingbird CLI has a simple install command that automatically configures a target for mocking.
$ mockingbird install --target Bird --destination BirdTests
Mockingbird is also easy to integrate into existing build systems. It only took a few lines of code to add it into our XcodeGen pipeline once more engineers started to use the framework.
Mockingbird runs as part of our build process, so it needs to be fast.
To measure performance, we developed a testbed of 1,000 source files, each with three protocols and multiple levels of inheritance.
Our benchmark shows that Mockingbird averages 0.5 ms per generated mock — 35x faster than solutions we initially considered.
We regularly profile Mockingbird’s generator and take advantage of concurrency and caching where we can. However, we got the biggest speed boost when we switched from Sourcery to SourceKitten, which provides a much thinner wrapper around parsing Swift.
0.5 ms per mock is great, but there’s still room for improvement. We’re constantly optimizing Mockingbird to make it even faster.
Robust code generation
Although generating mocks removes the need to write boilerplate, they can’t easily be modified by hand and need to handle all of Swift’s features.
Simple but expressive DSL
While our first attempts at code generation saved us the time normally spent on writing mocks and stubs, tests still included a lot of boilerplate.
Mockingbird needed testing APIs if we wanted engineers to adopt the framework, so we wrote an embedded DSL that builds semantics on top of Swift and hides the complexities of interacting with mocks and stubs.
We took inspiration from Objective-C mocking frameworks such as OCMock, and focused on making Mockingbird’s DSL easy to learn. The result is a compact DSL with only four key functions: mocking, stubbing, verification, and argument matching.
Mocking creates objects that can be passed in place of the original type and records received invocations.
let bird = mock(Bird.self)
Stubbing lets you define a custom value to return when a method is called or a variable is accessed.
given(bird.getName()) ~> "Ryan"
Verification lets you assert that a mock received a particular invocation during its lifetime.
You can use Mockingbird’s argument matchers when stubbing or verifying methods with parameters.
Since many of the standard library types conform to Equatable, Mockingbird automatically compares using equality and falls back to comparing by reference.
// The bird can eat fruits that are the same size as `apple`
let apple = Fruit(size: 3)
given(bird.canEat(apple)) ~> true
Mockingbird also provides wildcard argument matchers, such as any().
// The bird can eat any fruit
given(bird.canEat(any())) ~> true
Putting it all together
We can now use Mockingbird’s mocking, stubbing, and verification functions to test that shaking a tree containing a bird causes it to fly away.
The best part: Greater test coverage
Mockingbird now generates 100% of the mocks and stubs in Bird’s iOS codebase 🎉.
Even though our primary goal was to eliminate hand-written mocks, it turns out that Mockingbird’s biggest win is that our engineers are now writing more tests!