Optimistic Concurrency Control in HTTP Services

The Bestmile platform exposes several REST APIs that our partners and clients use to efficiently orchestrate their fleets of vehicles.

Some resources of our system can be modified by several clients. Consider the situation where two clients are working on the same resource, what happens if both clients attempt to update the resource at the same time?

Here is a diagram to illustrate the situation:

1. Alice reads the resource.
2. Bob reads the same resource.
3. Alice updates the resource.
4. Bob also updates the same resource.
5. When Alice reads again the resource, does she still see her changes? Or were they overwritten when Bob’s changes were applied?

One way to sort this situation out is to forbid concurrent updates: if the resource you want to update has already been modified since you last read it, your update is rejected. In our example, Bob’s update at step 4 would have been rejected because Alice modified the resource after Bob had read it.

This mechanism is called optimistic concurrency control. We can implement it by versioning the resources and requiring that a version is passed along with the update. We can use a simple version number assigned to 1 upon creation and incremented by each subsequent modification of the resource. When a client updates a resource, it also indicates the version it is working with. The server can thus validate whether or not the resource has been modified in the meantime. The diagram below shows how this would lead to rejecting Bob’s update:

1. Alice reads the resource, the server returns the resource along with its version number v1.
2. Bob reads the resource, he gets the same response and version as Alice.
3. Alice tries to update the resource, she sends the last version of the resource known to her along with her update command. The update succeeds because the current version of the resource on the server is still v1. The server returns v2, the new resource version.
4. Bob tries to update the resource, but his update is rejected since the current resource version v2 has changed since the last time he read the resource v1.

In practice, how do we handle the version of resources exposed over HTTP? it turns out that the HTTP protocol does have headers for this purpose. Indeed, a response containing a resource can use the ETag header to indicate its version. Conversely, a request updating a resource can use the If-Match header to indicate the last known version of the resource.

The remainder of this article shows how to implement optimistic concurrency control of resources exposed over HTTP with the library endpoints4s. First, we will explain how to describe HTTP endpoints that use the ETag and If-Match headers, and can return several possible responses (OK (200) or Precondition Failed (412)). We will then see the server-side implementation of these endpoints. Finally, we will see how to invoke the endpoints as a client, and how to publish their OpenAPI documentation.

Describing Endpoints

The system we want to implement has two endpoints, one for reading the current state of a resource, and one for updating the state of the resource:

The endpoint for reading the resource returns the resource content, and its current version number in the ETag header.

The endpoint for updating the resource takes the new resource content in the request entity, and the last known resource version in the If-Match request header. If the update is successful, the response status is OK (200), the response entity contains the updated resource, and the new resource version is provided in the ETag header. In case of failure, the response status is Precondition Failed (412) and the response is empty.

To implement this system with endpoints4s, we first need to write a description of the endpoints, from which, we can derive the server and client implementations, as well as the OpenAPI documentation. The library endpoints4s provides building blocks for describing endpoints, our first step consists in using these building blocks to describe the particular aspects of the endpoints we are interested in, such as “an ETag response header carrying a version”, or “a response that could be either OK (200) or Precondition Failed (412)”.

Let’s start with the description of an ETag response header containing the version of a resource. A first attempt is the following:

We create a trait VersionedEndpoints that enriches the base trait endpoints4s.algebra.Endpoints with a value describing the ETag response header. However, there is a problem with this definition: we get the raw content of the header, as a String. We can improve it by parsing the content of the header to extract just the version. To make it easier to distinguish between a String value that contains a raw ETag value and a String value that contains a parsed version, we also introduce a type Version:

The xmap operation transforms the ResponseHeader[String] into a ResponseHeader[Version] by using the two functions, decodeETag and encodeETag.

Our implementation is incomplete because it doesn’t support “weak” ETag values, but it is sufficient for the purpose of this article.

Similarly, we add the following definition to the VersionedEndpoints trait to describe an If-Match request header containing a version:

The next step is to describe an OK (200) response whose entity contains a JSON representation of the resource, and whose ETag header contains its version:

Since we want to represent our resource as JSON, we update the definition of the trait VersionedEndpoints to also mix in the JsonEntitiesFromSchemas base trait. The method versionedResponse describes a response with the status code OK, carrying a JSON entity and an ETag header.

To avoid manipulating a pair (A, Version), we create a type Versioned[A]:

Finally, we describe a response that can be either OK (200) or Precondition Failed (412):

Note that the result type of preconditionFailedOrVersionedResponse is Response[Either[Unit, Versioned[A]]], which means “a response containing either no value (in case of update failure), or the versioned resource”. An equivalent but slightly more practical way to model the response result would be to use the type Response[Option[Versioned[A]]] (where None represents an update failure). We can achieve this by using xmap again:

Now we can put everything together to describe the endpoints for reading and updating a resource:

As specified at the beginning of the article, the endpoint for reading the resource uses the GET verb and the /resource URL , the response contains the JSON representation of the resource in the entity, and the resource version in the ETag header.

The endpoint for updating the resource uses the PUT verb and the /resource URL , the request entity contains the new resource content, and the If-Match request header contains the last version of the resource known to the client. The response is either Precondition Failed (412) or OK (200). In the latter case, the response entity contains the updated resource content, and the ETag response header contains the new resource version.

Implementing a Server

To implement a server for these endpoints, we apply a server interpreter to the endpoint descriptions and provide their business logic. The logic for the read endpoint consists of reading the current state of the resource from the database, and the logic for the update endpoint consists of updating the state of the resource in the database.

Let’s assume we have a database with the following API:

This API would be easy to implement with slick-repo, but this is left as an exercise to the reader.

So, assuming we have an implementation for our Database trait, we can implement a server for our endpoints:

In this example, we apply an Akka-HTTP server interpreter by mixing the traits that are in the package endpoints4s.akkahttp.server to our trait ResourcesApi, which contains the description of the endpoints.

We use the operation implementedByAsync on the endpoints to supply their logic, which forwards to the database API. As a result, we get good old Akka-HTTP routes.

The server interpreter makes sure that incoming requests are well formed. For instance, if the If-Match header is missing from an update request, the server returns a Bad Request (400) response with a sensible error message.

Implementing a Client

Implementing a client consists of mixing a client interpreter to the trait ResourcesApi. For instance, to use a client based on Akka-HTTP:

And we are done. We can use the client to test the scenario presented in the introduction of the article:

Under the hood, the client sends HTTP requests according to the description we have written above, these requests are handled by the server, which returns HTTP responses, that are eventually decoded by the client.

Generating OpenAPI Documentation

To make our service usable for our partners, we publish an OpenAPI document for it. We can get an OpenAPI document for our service by applying an interpreter to our endpoint descriptions:

The OpenAPI interpreter provides an operation openApi that takes the endpoints to document as parameters (we pass read and update) and it returns an OpenApi value containing the documentation, which we can serialize into JSON by using the serializer provided by endpoints4s.

The resulting document can be processed by tools like SwaggerUI or ReDoc to build developer portals. For instance, by adding more documentation information to our endpoint descriptions we can generate the following documentation with ReDoc:

For reference, here is the complete endpoint description:

The complete source code is available here: https://github.com/julienrf/optimistic-concurrency-control-http.

Conclusion

Optimistic concurrency control is a solution for handling concurrent updates applied to the same resource without having to resort to a locking system.

We have shown how to embrace the HTTP protocol to implement optimistic concurrency control for resources exposed on REST APIs. Our implementation guarantees that servers, clients, and documentation are consistent together. Furthermore, it provides a clean separation between low-level concerns related to the HTTP protocol, and high-level business concerns.