API Versioning: Here we are, a complex backward compatibility is needed

The time has come!

Photo by Vishnu Mohanan on Unsplash

Until a few weeks ago, all our API breaking changes could be handled by transforming the payload or the request parameters. We knew one day we would have to tackle a more complex situation where it would not suffice. This article will explain how we improved our API versioning system to answer those situations.

The issue we faced

In late 2022, we delivered the V7 version of Whoz. This was a huge release targeting a great step-up in scalability, where we made drastic changes to our data model, among a lot of other things.

One of these changes was the addition of an organization level, representing the whole company and called federation, above our previous organizational root element called workspace.

This means that a good part of other entities now must have a federationId, instead of (or in addition to) the V6-required workspaceId property. That’s the case for the talent entity, representing a person in our model.

To ensure talent creation was still possible in V6, we had to determine the federationId corresponding to the workspaceId given in input payload. So a call to the /workspace API is necessary to fetch the corresponding federationId, our versioning system had reached its limits.

Generalizing the transformer interface

We built our API versioning system on top of Spring Cloud Gateway which is based on Spring WebFlux, and therefore is a reactive system. This means any blocking call is prohibited (if you aren’t familiar with Spring Webflux and reactive programming, this should help).

To call another API we had to use a reactive web client, so our previous interface based on Json bodies has been changed for a Mono (the reactive library we use is Reactor).

We went from this:

To this:

This obviously makes all our previous transformers' implementations violate the contract and break compilation. Rather than editing them all to encapsulate their json transforming code into a Mono, we decided to create a default implementation to the interface that does exactly that.

This allows all current transformers to remain untouched. Overriding the transformRequestObjectNode and transformResponseObjectNode methods are sufficient when only payload transformation is needed, which will also hopefully be the case for most future transformers, so the code of those basic transformers stays clean.

One downside of this approach is that compiler won’t force you to implement a method when creating a new implementation of VersionTransformer , since every method of that interface has a default implementation. In our eyes, this code design oddity is worth the gained clarity. Moreover, a developer implementing a new VersionTransformer would do it for a purpose, overriding nothing clearly wouldn’t fulfill that purpose.

Call another API to enrich the payload

All Right! Now that we allowed for reactive flux tinkering into our transformers, let’s implement our backward compatibility for V6’s POST/talent.

We need to

• Get the workspaceId from input payload
• use it to call GET/workspace/{workspaceId}
• Get the federationId in the workspace API response
• Put it into the /talent payload

That’s it for the request part.

We have nothing to do here for the response part, the workspaceId is still there, the federationId will be returned in addition to it, but this is not a breaking change (actually the situation was much more complicated than the example we describe, but this would add nothing to the point here).

Let’s look at the code.

Pretty self-explanatory, once you’ve got to know Spring Reactive. We use the WebClient and its methods to make an HTTP GET call and create a Mono from the retrieved body. After that, we just need to patch the talent request body with the federationId found in the GET workspace response.

You probably know about the YAGNI principle. In the case of API versioning, we actually knew we would need to handle complex compatibility one day or another.

Still, we went for the minimum we needed at the time. Should we have wanted to embrace all the complexity at once, the task would have been more difficult, and above all, we would have lost focus on the system design.

Keeping this focus, we have been able to design a flexible and rather simple system, that has been easily extended here to solve more complex issues.