Assertions in Production

Backend engineers know exactly what’s going on with their services. They know their QPS, memory footprint, error rate, and CPU utilization in real time. They even get paged automatically if there’s a critical error so they can respond quickly. Unfortunately, iOS engineers usually only have crash reporting and maybe some product metrics. Usually, we learn about bugs when a user reports it, or worse, when a user posts a bad review. We can do better.

Reporting Assertions

Assertions are conditional checks that ensure that your code is working as expected. Likely, you already have a few in your codebase. They check conditions which should never fail, and if they do, it’s signifies a critical bug in your code. However, assertions are removed in production and have no effect in the real world. On every project I’ve worked on, I’ve added a custom Log type which makes assertions report the error in production. Most crash reporting frameworks have the ability to report non-fatal errors, so implementing this is trivial. Here’s an example using Crashlytics:

enum Log {
    static func assert(_ condition: @autoclosure () -> Bool, _ message: @autoclosure () -> String, file: StaticString = #file, line: UInt = #line) {
    #if DEBUG
        Swift.assert(condition(), message(), file: file, line: line)
    #else
        if !condition() {
            let domain = message()
            let error = NSError(domain: domain, code: domain.hashValue, userInfo: nil)
            Crashlytics.sharedInstance().recordError(error)
        }
    #endif
}
    static func assertFailure(_ message: @autoclosure () -> String, file: StaticString = #file, line: UInt = #line) {
        Log.assert(false, message, file: file, line: line)
    }
}

In DEBUG, the assert works as it normally does and crashes your app. But in RELEASE builds, it creates an error and reports it to Crashlytics. This will show up like a normal crash but with a non-fatal label. It even includes a stack trace!

This has been invaluable to responding quickly to bugs. A new non-fatal exception tells me that my code is behaving in an unexpected way. With a stack trace, iOS versions, and the frequency of the problem, I can start trying to reproduce the problem and fix it. Now, I’m responding to bugs before users report them or write bad reviews.

Property Tests

I’d argue that this use of assertions is a form of property testing. To write a property test:

1. You define certain properties which must hold for a certain function.
2. Then, you feed the function random input.
3. If the properties hold, then the test passes.

With assertions, instead of writing these tests explicitly, your users generate the input! Your assertions define the ‘properties’ that should always be true. Though your users don’t generate purely random input, the input is often diverse and unexpected. The best part is — these tests don’t require writing any real tests!

Of course, relying exclusively on testing in this manner isn’t a good idea. But, it does give you some additional confidence that your app is running correctly. Source

If you’re interested in property testing, check out LayoutTest — a framework which enables you to easily write property tests for your views. The library generates random JSON data and inflates your views with it. Then, you assert properties on your view (such as spacing is correct and no views are overlapping).

When to Assert

Assertions are underused in most iOS apps and frameworks. Incorrect assumptions about how a function will be used are a common cause of subtle bugs . My general rule is:

If there is no else statement, there should be an assertion.

For example:

guard let x = x else {
    return
}

What if x is nil? Do we need to show an error to the user? Will some infrastructure silently fail? Often, x should just never be nil. In this case, I’d add an assertion to the else statement:

guard let x = x else {
    Log.assertFailure("x is unexpectedly nil")
    return
}

This is easy to add and also helps document assumptions in the code. Here, my assumption is clear: x should never be nil.

I also add assertions for:

• Verifying function parameters are correct
• Reading from dictionaries with ‘known keys’ such as NSNotification userInfo
• Log.assert(Thread.isMainThread, "Not on the main thread!")
• Opening a critical database or file fails

Shouldn’t I Let Assertions Crash in Production?

I know some of you are thinking: ‘when you hit an assertion, your program is in an unknown state, so it’s better to crash than continue.’ In theory, this may be true, but in practice, I’ve never found it to be valid. It’s often trivial to provide a degraded experience to the user. An empty screen is better than a crash. And since you’ll be able to react quickly to the problem, a fix will be out soon. If you are developing a security framework or banking application, crashing is maybe acceptable. But for most applications, it’s unnecessary to burden your users with your mistakes.

But, I Don’t Hit Assertions When Developing

You may not hit many assertions when you are developing, but you’d be surprised at what states your users manage to get your app into. You probably don’t test your app on every device, in every network condition, with every operating system, and with every account. I have seen many assertions fire in production which I’ve never seen before in development.

Are you using a different system to track errors? Do you have other ideas for improving assertions? Let me know in the comments!

Thanks to Sanket Firodiya, Alice Avery, Kamilah Taylor, Melissa Huang, Joe Fabisevich, and Kyle Sherman for helping review this post.