Maintaining API Versions in Go

This past weekend I spent some time working on a proof-of-concept system for building, maintaining, and testing versioned APIs in Go.

It’s called pinned and you can checkout out here!

Context

Services and clients (e.g. APIs and apps) need to agree upon how they communicate; they need a contract. A service expects a client to format their request in a particular way, and a client expects a service to respond in a particular way.

For an HTTP based API, the two main components of this “contract” are the endpoint and the payload; where to find/supply data, and what the data looks like. Any changes to this contract must be communicated to the client. If the client does not implement the changes, the app could break!

In cases where the service and client are used by the same party (e.g. an internal API), these updates can be done in parallel, and issues can be avoided. However, in majority this is not the case. For example, if a user doesn’t update their mobile application. If the service provider wants the user to continue using their app they must maintain the “contract” set at that time.

This “contract” is an API version. And their are two popular ways to do this.

Endpoint Versioning (e.g /v1/posts)

• Facebook
• Twitter
• GitHub

Date Versioning (e.g. /posts?v=2018-02-11 or Header: 2018–02–11)

• Stripe
• Foursquare

In both cases, the client is “pinned” to a certain version. If the client upgrades (app update, website improvement), this version may increment. Typically, major API providers communicate the lifecycle and duration of a version to their consumers. This enables the app owner to prepare for breaking changes.

Problem

As the number of supported versions increases, the complexity of improving an API increases as well. This is taxing on the development process, sucking time away from new improvements to legacy maintenance. Although forward thinking can be helpful, breaking changes are inevitable.

This poses two challenges:

• Maintain old versions without cluttering/blocking new developments.
• Update our testing to ensure old versions work.

Stripe published a blog post on how their approach to the first part, which I’ve used as inspiration.

Solution

To maintain, keep track of versions and supply methods for migrating specific resources between versions. To test, take snapshots of each supported version, and compare against when testing.

Version Migration

This is solved by pinned. Consider an example API with the following versions and changes:

2018-03-09 - Rename User.FullName -> User.Name
2018-02-09 - Add User.CreatedAt
2018-01-09

A client requesting 2018–01–09 will run into a breaking change when the field name changes in 2018–03–09.

How might we solve that right now? Out of the box we can do something like this (pseudocode):

func getUserRoute(req, res)
  switch (req.version) {
    case "2018-03-09":
      user["name"] = user["full_name"]
      break
    case "2018-02-09":
      user["created_at"] = user.date()
      break
    case "2018-01-09":
      break
    // ...
  }
  return res.write(user)
}

In an endpoint we check against every version, make appropriate changes. For a couple versions, this isn’t too bad. But as the number of versions increases, the majority of a route’s logic becomes resolving versions. And as the number of routes and resources increase, it becomes particularly complex to understand how an improvement specific to one resource, affects multiple routes, and multiple versions of those routes.

pinned resolves this by enabling a service to declare a “version.” The version is composed of a date (to enable easy comparison and ordering) and a set of changes.

These actions work in reverse; they migrate resources from the it’s version to the previous. It tells pinned how to “undo” the changes made for that version. In this example, userNameFieldChange is a function which migrates any instance of User from 2018–03–09 to 2018–02–09.

Using our example, a request made for 2018–01–09 to an API who’s latest version is 2018–03–09, will execute the set of migrations from the two versions ahead of it, making the latest compatible with the earlier version.

What’s so great is that the developer can now focus on new development with confidence that older versions will be supported.

Testing

This all sounds great, but an increase in the number of versions also increases the complexity in testing. Tests would need to check against every version, and since responses evolve over time, each test would have to keep track of those response formats as well.

I’ve previously released and written about a project called abide which uses snapshots to make API testing very easy.

In conjunction with pinned, an example test against all versions of an API looks like:

The snapshots generated look like:

Now, any improvements / changes to the API can be tested against all previous versions of the API automatically, without having to write new tests for new versions.

Conculusion

This is still mostly a proof of concept and there is more work to be done. However, I believe the combination of (1) writing migrations and (2) using snapshots is effective for building, maintaining, and testing a versioned API.

All feedback, issues, PRs, contributions, etc. are more than welcomed! I’d like to see where this project could go!

Feel free to drop me a line @stevekaliski or sjkaliski@gmail.com 😃