Web APIs are all the rage today because the internet, right? Behind all the hype behind this three-letter acronym lies this great holistic concept: you can create great service by combining functionality that resides elsewhere.
All this is great, until the moment something breaks. Highly interconnected systems are susceptible to downtime and breaking changes on third-party services. Today’s post is about dealing with the latter, from a service producer and consumer perspective.
Not all changes are born equal
So, let’s start with this: in a modern API, not all changes are breaking ones. Mike Amundsen has a great talk touching that subject, in which he brings this pattern:
By following that simple principle (all the other ones are good to — you should watch that talk, I’ll be waiting here), an add-only modification has zero impact on clients. This encourages designers to be even more minimalistic in their initial interface contracts: since adding fields is non-breaking (not always, see below), deferring their use is usually a smart move.
Now, things can and will get ugly on occasion, as you may be forced to do changes that directly impact your clients. Such changes may be:
- removing fields and operations: a no-brainer here, so moving forward.
- adding constraints: limiting acceptance on input fields are breaking changes as well. These include: making a field mandatory, setting up a min/max length limit, etc. This is a classic database refactoring challenge as well;
- softening constraints: counterintuitive, perhaps, but softening an output constraint may break a client that is still enforcing it. If a given API starts allowing a field to be null, it may break a client that still deems it mandatory.
- semantics: this one is frequently forgotten and tends to be a nasty case. Changing the meaning of data (“addresses are now formatted this way”) or the possible range of values (“I am now considering a new possible status for orders”) are good examples of such situations.
Reconciling a potential conflict
First of all: API breaks and continuous deployment are not the best of friends. A public API needs a fair amount of planning before releasing a breaking change which can cause development teams on the producer side to start stocking unreleased features.
So, on one side, you may have a team that wants to move fast and break things on the producer side. On the consumer side, you may have a team that wants to move fast as well, without being constantly impacted by someone else breaking them.
As seen above, API changes may be released without any breaks, as far as the producers follow an add-only pattern and consumers follow a must-ignore one. And in those cases, producers should release, as frequently as they want, and their sole concern on API versioning should be the bumping of their SemVer minor version.
This approach can be leveraged for a database-refactorings-inspired flow: most breaking changes can be divided into smaller, digestible steps. For example, let’s say you are introducing a new mandatory field:
- Communicate the change you intend to do, describing the steps below and their timelines.
- Introduce the field without any constraint. Assign it a default or calculated value if possible (will help with step #5).
- Start monitoring: all clients under active development should gradually take the cue from step #1 and start providing that value over time.
- Optional: follow-up with clients who are lagging behind.
- Start filling your database with values for the new field (kudos if you were able to use a default/calculated value, as you will not be running against a moving target).
- Introduce the breaking change (don’t forget to bump your major version). If all clients have converged, you can simply deploy the changes. If there are clients who haven’t, you may have to support one more version of the API.
An important point to keep in mind, then:
The add-only design approach means, among other things, that developers should be even more obsessed with the creation of minimal interfaces.
After all, if new fields are required, this change comes at a much lower cost than removing unnecessary/undesired info. It is such a case that you will see, on occasion, interfaces with vestigial structure, like the appendix we still carry, left there for old times sake.
Long story short:
- Do not introduce API breaks unless you positively, absolutely have to — there are usually ways to avoid it.
- If your API is public, make sure you communicate the change, along with a generous timeline.
- Consider refactoring your way into the change, if possible.
- Due to complex change management, interface design is one of the few points in which you want to pay a bit of up-front design.
This post barely scratches the surface of a multi-faceted problem. You may have noticed that there was no mention of versioning APIs, ways to detect breaking changes, and how to deal with external dependency downtime (all in its due time).
In writing this, I held the urge to go into details about potential breaks, for brevity sake, but this still felt like people could benefit from a more detailed breakdown. Thus web-api-changes-catalog was born. It is by no means a complete list of potential breaking changes right now so please do send your feedback (or even better: PRs).