Validating JSON Patch Requests

HTTP PATCH requests are pretty straightforward. Validating them is not. I spent a lot of time trying come up with a clean strategy to validate the results of RFC 6902 (aka JSON Patch) formatted REST Patch requests. Here are the results of my findings and solution in Node.js.

Background

I am not going to give a background on what a PATCH request is, so take a look at William Durand’s (unfortunately named) article, “Please. Don’t Patch Like An Idiot.”

Goals

• Prevent modification of “immutable” fields (e.g. ID, date created fields)
• Prevent users from adding new fields where they don’t belong (top-level fields, etc)
• Provide feedback to users when they have violated a condition
• Do not implement a new validation tool
• Provide a Node.js solution

Constraints

Database drivers generally do not natively support the JSON Patch spec (or not that I have seen). DB drivers do generally provide mechanisms to update either a single field or to update an entire document. I ruled out making multiple update requests against the DB for each JSON Patch object, and instead decided that all checks would have to happen server-side and provide the complete updated object to the DB.

Use Case

Let’s say that I have this document:

{
  "id": "c1234",               <-- ✘ non-modifiable ID
  "name": "Shark",             <-- ✔ modifiable name
  "meta": {                    <-- partially modifiable metadata
    created: 1452474481612,    <-- ✘ non-modifiable timestamp
    color: "red"               <-- ✔ modifiable color
}

… and I want to apply this JSON Patch:

[
  {"op": "replace", "path": "/id", "value": "foo"},
  {"op": "add", "path": "/description", "value": "A red shark"},
  {"op": "remove", "path": "/meta/created"}
]

There are 3 things going on here:

• ✘ Replace the “id” fields with “foo”. This will not be allowed.
• ✔ Add a “description” to the root of the document. This will be allowed.
• ✘ Remove the “meta.created” field. This will not be allowed.

Poor Approaches

There are a number of approaches that I tried that did not work. They are listed here in the hopes that others will not waste time trying them.

If you do not care about things that don’t work anyway, skip to the section titled “A Great Approach”.

Poor Approach #1: Apply the Patch and Validate the Result

At first glance, this seems like the most obvious solution to the problem. The idea is to take the original document from the database and simply apply the patch to it and validate the resulting object.

This approach works really well if you only need to worry about a field or two. However, this approach falls apart in three critical areas:

• The developer must now keep track of not only illegal fields but also of field modifications.
• Telling users what is invalid can also be difficult because of the prior point.
• It gets even worse with deeply nested objects/arrays and recursion.

For example, given this document:

{
  "id": "c1234",               <-- ✘ non-modifiable ID
  "name": "Shark",             <-- ✔ modifiable name
  "meta": {                    <-- partially modifiable metadata
    created: 1452474481612,    <-- ✘ non-modifiable timestamp
    color: "red"               <-- ✔ modifiable color
}

… applying this patch

[
  {"op": "replace", "path": "/id", "value": "foo"},
  {"op": "add", "path": "/description", "value": "A red shark"},
  {"op": "remove", "path": "/meta/created"}
]

… produces:

{
  "id": "foo",                    <-- ☹ ID modified
  "name": "Shark",
  "description": "A red shark",   <-- ☺ added description
  "meta": {
                                  <-- ☹ timestamp removed
    color: "red"
}

In order to validate this, the validator needs to know about:

• The fields that are required, forbidden, or optional
• The current object being validated
• The original object from before the patch was applied.

And to top it off, if a patch were to be applied to an array of nested objects, we would need to know what is allowed to be modified and what is not, and if anything is not allowed to be modified, we would need to run a diff on the object/values, which could be nested.

If you don’t believe that these points are difficult to validate, I encourage you to validate an object as compared to its changes with any moderately complicated schema. ;)

Poor Approach #2: Validate Each JSON Patch Object

One my initial ideas was to iterate through every JSON Patch object and check the object against a set of rules. This means that the “op”, “path” and “value” fields in the Patch must all be checked against white and black lists. Coming up with an all-inclusive white/black list is possible, but would suffer from high maintenance costs when schema changes are made, and ensuring that 100% of the cases were covered could be a daunting task for any mildly complicated schema.

Similar to the white/black list approach, attempts to validate paths with RegEx and glob patterns produced patterns that were much too complex. The patterns would have to forbid unknown fields in one object and allow them in another. The patterns were both hard to write and hard to maintain. The final nail in the coffin was the inability for the patterns to perform type checking on the values themselves.

It also doesn’t help that I and many others I know have a difficult time writing RegEX patterns…

Poor Approach #3: Build an Object up from the Patch

One failed approach was to start with an empty object and apply the patch to the object. When fields were added or replaced, the “value” field of the patch would be added to the object. When a field was removed, it would be replaced with undefined. After the object was built, it would be compared against a schema where all the fields were optional.

This approach was flawed because it attempted to take a perfect JSON Patch document and transform it to a different format that produced things like arrays with holes.

{
  "array": [ , , { nested: true, deleted: undefined } ]
}

This approach also fell apart when trying to represent deleted keys in a clean way. Doing this correctly would require at least a partial implementation of a validation framework to cover cases such as fields being deleted while maintaining type and data checks.

In short, this is a really dumb approach.

A Decent Approach

One decent approach that I developed uses a few steps:

1. Pull the original object from the DB
2. Apply the patch to a copy of the original object
3. Remove the fields that are not allowed to be modified from both the original and patched objects
4. Diff the original and patched object
5. Compare the resulting comparison to the original patch.

In a nutshell, these steps check that the object being modified by the JSON Patch is modified only in ways allowed by the validation schema. It achieves this by comparing the actual differences made to the source object to the deltas defined by the JSON Patch request, which must match completely to be considered valid.

Code: https://github.com/MarkHerhold/json-patch-validation

Weaknesses

The implementation for this is moderately complex and likely not very efficient, as it involves cloning the original and patched objects many times.

Other Thoughts

This approach or one similar to it has the best chance of being ported to another language.

A Great Approach

The simplest approach is often the best approach. See it in action: https://github.com/MarkHerhold/json-patch-joi

This approach adds the noChange() function to Joi by monkey patching it in. You can get the plugin via npm or GitHub.

• npm: https://www.npmjs.com/package/joi-nochange
• GitHub: https://github.com/MarkHerhold/joi-nochange

The noChange() plugin checks that the value in the object being validated (the patched object) matches the value at the same corresponding path in a sister object (the original object).

Example

Here is an example of the end result:

JSON Patch validation using Joi with noChange()

Caveats

• This validation strategy allows patches that modify a protected value so long as it is “updated” to the exact same value. I consider this an unfortunate but acceptable risk.

Happy Patching!