Failing forwards with backwards compatibility
Designing an ever-changing app that always works
Somewhere in my house, I’ve got a couple of those old pound coins sitting around, tucked just out of sight. Quite why I still have them I’m not sure — kept perhaps out of a sort of nostalgia — but it’s at least nice to know that if I ever want to, I can still take them to a bank and cash them in. Even though the deadline has passed, and these coins have been replaced by newer, shinier versions, they can still technically be used.
Allowing a gentle transition such as this is a very important part of upgrading or changing a system. Whether we’re talking about large things such as changing what constitutes legal tender, or smaller things such as updating a mobile app to the latest version, the same is always true; it should work for those who transition immediately, but it should also support those who lag behind. Enter backwards compatibility.
What is backwards compatibility?
When was the last time you were forced to upgrade — or worse yet, fully reinstall — an app on your phone because the version you had was suddenly unusable or broken in some way? Likely never. Most apps operate on the same backwards compatibility principles, that older or legacy versions can still interact with their system as a whole.
It’s easy to assume that each version of an app is an isolated piece, and independent of anything else. But when you take into account that every app version is likely fetching information from the same API, or recording data to the same database, the importance of backwards compatibility and supporting older versions becomes clear.
At Birdie, we provide an app for care workers to log and record their visits with elderly care recipients in realtime, allowing for instant access to care and visit information. In practice, this means we need to support a wide range of app versions sending information to our API and for those app versions to understand how to render any data it gets back from that same API.
Thus, any new feature that involves changing the response from the API becomes an exercise in backwards compatibility.
Let’s look at a Birdie example. Birdie’s app has the concept of a visit log inside it, on which you can view any given visit that has been recorded by a given caregiver.
Up until recently, our system has only had one caregiver per visit. Now, however, we support “double-up” visits, which could theoretically involve an infinite number of caregivers completing a single care visit.
So, we have our caregiver data returned from our API to the mobile app, which holds information on the single caregiver that completed the visit.
But now, some time later down the line, we find that we need to update that to return multiple caregivers, perhaps using a caregivers field.
Seems like all that needs to be done is update the API to return a list of caregivers instead of a single caregiver, and release a new version of your app to expect a list of caregivers instead of just one, right? Sadly, no.
This change means that older versions — which still expect a single caregiver— will request data from this same endpoint and crash upon attempting to use a caregiver field that doesn’t exist.
This behaviour is fixed in time, it cannot be undone; that version of the app will handle data in that exact same way, forevermore. The user experience will immediately be broken for every single person using any app version prior to the latest one — so how do we fix this?
Building, backwards compatibly
Although the transition from our old data model to the new one went well, it wasn’t flawless.
Without going into too much detail, adding the feature involved updating our model of visits to go from having a caregiver field to having a caregivers field. To support this, we took a stepped approach.
First, we migrated the data in our database table to store the list of caregivers, which would contain the single original caregiver.
Then we added a backwards-compatibility function to wrap all responses from our given endpoint, to make sure it returned a combination of the new and the old fields:
This allowed it to support both old and new apps, but it increased the size of the data unnecessarily.
Next, we started changing the app to expect the caregivers field instead. Once we were confident that the version we were working on was no longer reliant upon the older caregiver field, we released it as the latest version of the app.
With the latest release now using the new field, the last thing to do was tidy up. Leveraging the fact that we could determine the user’s app version when they made a request to our API, we were able to ensure that our backwards-compatibility function ran only for app versions earlier than the latest release.
Both the old and the new apps were now receiving only the data they expected, and we’d managed to transition to this new model without breaking anyone’s app experience.
Or so we’d thought. Unfortunately, we’d fallen into a particularly dangerous trap. We hadn’t tested it by using it exactly as a user might.
What we missed
Our app is designed to work both online and offline, allowing a user to load information whilst online and then read and record information as they go, even whilst offline, which is then queued and sent to the server as soon as the device reconnects.
What this meant, in our case, is that the data fetched from the API is stored locally on the device in a redux store, allowing the user to continue working offline with the app as normal.
This presented a fundamental flaw in our backwards-compatibility plan.
All the user had to do was fetch data from the app on an older version (thus storing it in their local redux store), update to the latest version on the app, open it up again and boom, the app would crash.
The newer app did not know how to handle the older field passed to it from the redux store, nor what to do without the newer fields, and so did not work.
Our learnings
Ultimately, we failed forwards, and did our best to fix it, but it came as an important learning for me.
Backwards compatibility is easy to overlook, and difficult to get right — even when you think you’ve covered everything, think again.
On top of that, always always test your app by using it as a user might behave. Mess around with it, use it as expected, and then upgrade and continue to use it further. Had we tested like this, we likely would’ve caught this bug in advance.
Of course, not everything can be maintained forever, sometimes you need to stop supporting the oldest versions in order to move forwards. The important part is to encourage your users to move with the times and keep relatively up to date.
After all, the town bank can’t accommodate the bloke that shows up with a bag of shillings and half-pennies.
We’re hiring for a variety of roles within engineering. Come check them out here:
