NetNewsWire Crash Assessment

tldr; Unneeded check + harmlessly expired cert + live assertions:

The combined effects of three different factors have resulted in the fact that NetNewsWire is now crashing on launch.

Please note: this was discovered last night and a fix was tested and submitted to Apple immediately. Our first ever request expedited reviews was granted for both the Mac and iOS versions of NetNewsWire.

The iOS update has been approved and should be on the App Store now.

We’re still waiting for the MAS version of NetNewsWire to be approved, but the direct download version of NetNewsWire for OS X was updated last night and can be downloaded by owners of the direct version of NetNewsWire.

Since NetNewsWire for OS X can be unlocked via automatic detection of MAS receipts it will also work for people that bought NetNewsWire from the Mac App Store.

Download link: http://netnewswireapp.com/mac/download/

Root Cause Analysis

1. NetNewsWire uses certificate pinning as a part of the security model of its sync service. A copy of the public SSL cert is bundled with the application and is used as a source of the public key we use with the sync service.

2. Client side code used an assertion to validate the bundled cert before using the public key. This check was unnecessary as the public key would still be usable and correct regardless of whether or not the local cert had expired.

The spurious requirement for a non-expired cert in the bundle was completely out of scope for the application and would require a scheduled application update each year just to renew the cert on the device. This was never a part of the plan or an expected requirement for the roll out of NetNewsWire on any platform.

A failure to update the bundled cert with this check in place would mean that the app would no longer sync with our services once its local cert had expired. Sync would not work until the app had been updated.

3. The 2015 cert we use for NetNewsWire expired. The NetNewsWire cloud sync servers were updated with the renewed certificate, but, not realizing that the validity of the client-side cert was mandatory due to the unexpected check, the client app was not updated as well. It was believed that updating the servers would be sufficient to keep things running smoothly.

Given that expired state of the certs DID matter, the expected outcome of this situation, as written, would be that all installed instances of NetNewsWire would stop syncing. We would panic, fix the issue, submit the builds to Apple, pray for a fast turn around on the review and ride out the surge of upset customers and hope to redeem ourselves.

Unfortunately, this is not what happened. What did happen was far worse.

One More Thing

Assertions are used by developers to catch heinous problems during the development process. The rationale is that the situation is dire enough to warrant killing the app outright and leaving a stack trace so that you are forced to identify the problem and fix it before shipping.

Assertions are turned off for the builds people hand to their customers because bursting into flames is considered to be a poor user experience.

Somehow, assertions were enabled in the NetNewsWire builds that were out in the wild. This is a Very Bad Thing. We are still investigating how this came to be, as all of the configurations for Release and Debug builds look as we would expect them to be (NS_BLOCK_ASSERTIONS=1).

The result

All versions of NetNewsWire would superfluously check the validity of the expired local cert before using the public key, then draconically crash the app when the validity assertion failed. There was no way to fix this without updating the app itself.

We were able to quickly assess what was causing the problem and removed the validity check, regression tested the apps against the sync server, and then submitted our builds.

Now we wait.