A better way to implement HTTP PATCH operation in REST APIs

Photo by Natalie Rhea on Unsplash

Although there are a lot of HTTP methods, there are four methods we mainly use in REST APIs. Namely; GET, POST, PUTand DELETE. Those HTTP methods represent common CRUD operations that can be done on top of a resource that we expose to the outside world.

To update a resource, we should use PUT. But also you could use PATCH as well. The difference between those two is that PUT fully updates the resource, while PATCH updates it partially. Therefore PUT is always considered as an idempotent operation while PATCH is not (in some scenarios).

While all other operations are trivial, I have seen developers are implementing these PATCH methods in different ways. Sometimes PATCH has been ignored in some APIs, and instead, asking clients to use PUT for the update of resources. You can simply identify these different patterns by inspecting the API contract of a patch endpoint.

To explain the different implementations, I will take the User resource as an example. And also obviously, for better clarity, I have ignored all the getters/setters.

User resource representation

So, what are the ways of implementing PATCH operations?

Method 1: With Full Resource Object

In this way, the webserver expects a client to send the full resource object including all the changed and non-changed fields like they should have submitted to the PUT endpoint.

For e.g. In the above example, a client should send the full User object with all fields. Once received, the webserver can do two different things.

1. The server will figure out the diff between provided and already persisted data, and then will apply the changes to the persistence layer accordingly.
2. The server will just act similar to the PUT operation where it will replace the whole document as it is.

The problem with 2nd approach is, it is still a PUT operation disguised as a PATCH operation. And importantly, there can be constraints that some fields may not be able to update again once created, or some fields might be derived values too. For e.g. considerusername, createdAt and updatedAt fields. We should not accept those values again or let alone the client updates them. Therefore, the server must carefully implement behavior that ignores such fields and updates the remaining part of the resource. So, 2nd approach effectively falls back to the 1st approach.

1st approach derives the object diff by the server itself and applies the changes. Again, this sounds trivial, but it’s not. Imagine how cumbersome is writing this diff identification logic across so many different types of resources and their unique fields? (lots of if (<field> != null) and so on) With the increase of the total number of fields in a resource class, this quickly becomes a nightmare.

Although this approach is very commonly used in industry, it is not scalable when the resources are supposed to evolve over time.

Method 2: Specific Diff Schema

In this method, the service will define a separate schema class for the patching of each resource. This new schema object closely resembles the original resource specification, but notably drops out all unmodifiable fields or derived fields. So, a client can update only the fields as specified in the schema. None other.

An example patch contract, that might be defined by the service, should look like this as shown below for our example User object.

This looks ok, right?

Yes and no.

But, Why? you might ask.

In this case, when it comes to the multi-value fields, those still needed to be provided fully by the client. Check the array of list fields like secondaryEmails. Still, the client has to provide all existing emails + added or removed emails. Hence this is still not truly a patch representation, but very close enough. To overcome this, some developers use a kind of variation.

Now, this is indeed a patch representation. It represents a diff.

But, as you already figured it out, this could get easily out-of-control, if the so-called resource has many multi-value fields.

Ideally, in REST, the service should have separate sub endpoints for each association. But developers/leads tends to ignore such granular resource management, mainly because of inability to handle atomicity of updates, and also the overhead causing in implementing such behaviour in a client.

Meanwhile, creating different representations for each resource of service could also become an overhead in terms of maintainability. While evolving, every time a field is added or removed from the original resource class, these patch classes may also have to be modified too.

This is another most common method I have seen using practically. However, as you have already seen, there can be an overhead of maintaining a completely different subset of classes/schemas for patching purposes. And also, still, developers need to explicitly find the client-defined fields (by checking for non-null fields) and hence the diff to proceed. Or similarly, they could merge this received object with the existing persisted object but only with non-null values.

Now, what if I say that there is a much better way to achieve this operation in a very generic way.

Method 3: Patch Operations

This is a specification published in IETF https://datatracker.ietf.org/doc/html/rfc6902 which provides an approach to support PATCHing in a more generic and efficient way. Operations are; add, remove, replace, move, copy and test. I would encourage you to read this specification because it is very short and has examples describing the methodology.

Each patch operation has a mandatory op field, while optional value, path, from, and to fields. The field path must adhere to the JSON Pointer specification which is very similar to XPath in XMLs but in a simplified manner. If you are using FasterXML Jackson v2.3 or higher in Java, then this is automatically available for you.

A PATCH specification accepts an array of such patch operations and applies them one by one on top of the specified resource. op defines the operator, and all other fields can be considered as operands. Note the whole PATCH operation won’t be successful if at least one operation is failed on the executing resource. That means the operation should be atomic according to the IETF specification.

Each entry in array has a sound like, “do this operation on this field with this given value(s)”.

[
   { "op": "test", "path": "/a/b/c", "value": "foo" },
   { "op": "remove", "path": "/a/b/c" },
   { "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
   { "op": "replace", "path": "/a/b/c", "value": 42 },
   { "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
   { "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
]

Since the specification is generic, you will have the same PATCH contract across every resource in the service. Note that, the headerContent-Type should be set to application/json-patch+json media type to indicate that it accepts a set of patch operations, but it’s not mandatory. You can have it as application/json and still, it will work.

For example, adding a new secondary email to the end of the array and setting the same email as the primary email in a User resource can be done in the below way.

[
   { "op": "add", "path": "/secondaryEmails/-", "value": "mynewemail@domain.com" },
   { "op": "replace", "path": "/primaryEmail", "value": "mynewemail@domain.com" }
]

Changing the order of the above operations does not impact the final output, because as mentioned above, the operations are independent and are atomic. But there can be situations where order matters, especially when you decided to use move and copy operations. So make sure your client provides make-sense ordering of the operations to the service.

Further refer the JSON Pointer specification to understand how path field can be constructed in different situations. And also some examples are there in JSON Patch specification as well at the end of the document.

Now let’s consider the advantages and disadvantages.

Advantages:

• Easily scalable to any resources and any number of fields of a resource. Hardly ever a spec change in endpoints again.
• Patching is self-explanatory. Gives a clear visual cue in logs when troubleshooting what is going on. (unless of course, as long as those are not sensitive data)
• Convertibility: The ability to convert a patch operation entry to a direct DB patch update, if the database supports such behavior, is an advantage. Databases such as MongoDB have this capability to some extent. But in this case, better to do all changes inside a transaction to provide the atomicity.
• Performance: In some scenarios, the provided test operation is very useful in terms of validating object values before or after operations, so the client does not need to fetch data and validate on the client-side, but it can be done in-place where the patch operation actually occurs (i.e. in the server).

Disadvantages

• Still, the specification does not prevent updating non-updateable or derived fields similar to in our Method-1. This validation needs to be done manually before starting to run all given operations.
• Now the client-side still need to derive the diff. But I don’t see this as a much disadvantage as generally clients (Frontends, users) are doing operations in this natural way that clients can track their changes as operations similar to the above-specified spec.

It does not mean that we are not allowed to be innovative…

Although we can strictly follow the IETF JSON Patch standard, we can also follow it loosely, and be creative depending on the situation.

As I mentioned earlier, the Content-Type header still can be in the normal application/json format and the functionality should still work. (See my example Springboot application I created)

There is another enhancement you could do if you can relax the strict atomicity requirement. With the strict atomicity in place, if one operation fails in the middle (maybe due to concurrent modification for the same field in the same resource), the client has to retry again with all the entries once again.

We can relax the atomicity by allowing a client to retry again only with the failed or not-executed operations again. To achieve this, we could introduce a unique id for each operation, and modify the error response in such scenarios to send successfully and failed (or not tried) operations separately as shown below.

PATCH /users/{userId}
Request: [
  { "id": "<unique-id-1>", "op": "add", ... },
  { "id": "<unique-id-2>", "op": "remove", ... },
  ...
]
200 Response: {
  ... // full user object
}
// In case of a document conflict
409 Response: {
  "success": ["<success-id-1>", ...],
  "failed": ["<failed-id-1>", ...],
  "notTried": ["<pending-id-1", "pending-id-2", ... ]
}

Adhering to the above way may cause to do a change in the client-side as well for repeating with remained set of operations. It is not a bit of good advice to do partial operations in API endpoints, but someone can argue that being a PATCH endpoint, it can be considered as a true patch in the above scenario. So, consider it wisely.

Here is a sample implementation that was done on top of SpringBoot. https://github.com/isuru89/springboot-patch-operations

Side Note: Although I have exposed DTO classes directly through the controller layer, in an ideal world, it is discouraged and you should not be doing that considering security reasons. You should have a separate set of POJO classes for each resource. The example is for just demonstration purposes only.