Normalized GraphQL Caching in Swift
At Ibotta, we’ve spent the last year fully rewriting our graphQL networking code to transition from Apollo to a custom solution. We’ve moved away from Apollo’s prebuilt query system to dynamically generated queries with our own normalized cached. Normalized caching enables very precise cache updates and allows cache lookups based solely on object type and id. This allows us to get cache hits on previously unseen requests.
Caching
Caching, and more specifically cache invalidation, is a really hard problem. There are all sorts of external events that could make a cache invalid. Even if you know what changed, it can be hard to mutate or purge specific elements of a cache. This article will go over a normalized caching strategy for graphQL that has helped us maintain reliable and accurate cache hits.
Let’s say we make a request for several offers and they are returned as:
We can cache the entire response to disk keyed by the request. This is a very common and simple caching strategy, but it is fragile and requires constant purging on mutations.
Let’s say the user later unlocks Offer.1. Our cache is now incorrect and needs to be purged. We would need to purge all cache entries that contain Offer.1 or maintain a separate data source to correct our models on cache reads. Neither of these options are ideal. The ideal solution would be to mutate our cache so subsequent cache pulls would return correct information.
What if we could actually mutate our local cache? Introducing normalized graphQL caching. All graphQL types have a __typename field and most types have an id field. We can use this information to create deterministic object keys.
We can then store the above response as normalized data where child objects are extracted and replaced with references. This looks like:
With this approach, if a user were to later unlock Offer.1 we could perform the graphQL mutation on our local cache. All we would need to do is create a cache key (~>Offer.1) and merge the updated properties (["unlocked": true]) into our cache.
Implementation
To store JSON payloads, we will need to separate out child objects; we will call this decomposition. And then when retrieving cached payloads we need to recompose the objects. The code in this article is going to be kept relatively simple and focused on recomposition and decomposition.
The interface for our cache is going to be very light weight: JSON in/out, cache clear, and object update.
The underlying storage for our cache will be an in-memory dictionary. However, in production, we use a thread safe LRUCache. We are going to start with a GraphQLCache implementation like:
Decomposition
To decompose the JSON into child objects, we are going to recursively parse the JSON and extract child objects when found. But first we are going to need a couple of helper functions. We need to be able to determine if an object has a normalizable key. If an object doesn’t have a __typename and id we can't extract it. We are going to prefix all our object references with "~>".
When storing normalized objects we might already have some fields of an object stored. So we need to be able to merge new entries into existing ones.
JSON can be categorized into three types: [Any], [String: Any], and Any where the last Any is for primitives. An [Any] could be an array of primitives or an array of dictionaries. With this in mind, normalizing the JSON is relatively straightforward.
To cache the top level request object, we will wrap our decomposeAndCache(object:).
And just like that, we have JSON responses being stored as normalized objects. But now how do we get our JSON back?
Recomposition
Recomposition is very similar to decomposition. We recursively parse the object and look for reference keys. Then we pull the objects the reference keys point to and recursively recompose those objects.
However, if an extracted child object was removed from the cache, we need to fail recomposition. If we have an array of references like ["~>Offer.1", "~>Offer.2"] and ~>Offer.2 is missing from the cache, we need to throw and fail recomposition.
Recomposition should take an input like ["offers": ["~>Offer.1", "~>Offer.2"]] and return:
To use our recomposeCached(object:) function, we can provide a clean public function.
There is a risk of cyclical cycles occurring during recomposition, but in practice we do not see this very often. There are a few options to get around it; the naive approach is to specify a max depth for recomposition. Or you can keep track of recomposed object keys and stop recomposing when a cycle occurs.
Cache Times
In the real world, cache records have to expire. One of the nice things we can do with a normalized cache is vary the cache times based on object type. We can cache an Offer for one hour and a Retailer for several hours. What's more, we can normalize the cache time on a per field basis. Let's store cached entries with timestamps, like so:
This incurs some extra storage costs, but in graphQL we often fetch the same object with various fields so we’ve found it was worth it. To create field times, we can do:
Next, we can update our decomposition function like:
Then, when using the cached JSON, we can evaluate that the cache time hasn’t passed. We can either evaluate cache times on a per field basis during decoding or build in the evaluation into recomposition by finding the oldest cache time and using that as the object’s cache time. We aren’t going to get into the nitty-gritty of how to edit the recomposition code to support cache times, but it should be pretty straightforward. At Ibotta we evaluate cache times on a per field basis during decoding.
Cache Updates
Now that we have this powerful normalized cache, it’s time to actually utilize it. One of the key reasons to use a normalized cache is to allow easy updates. We can retrieve cached objects based on typename + id, and then we can manipulate any of the properties.
On updates we aren’t going to update our field timestamps because we want to treat the timestamps as when we were last guaranteed accurate data from the server. Assuming we had hooked the cache into our networking code, if we wanted to unlock Offer.1 from the first example, we could do:
You could also delete particular elements and not have to wipe huge chunks of your cache.
Conclusion
The code presented in this article is a great starting point for building a robust and powerful normalized cache. When making a request for objects for ids, we can check our normalized cache for those objects and potentially get a cache hit for unseen requests. Additionally, because we are merging in duplicate objects, stale objects could potentially get updated by unrelated requests.
At Ibotta, we use this normalization strategy in production with great success. By being able to mutate our local cache, we see almost a 60% cache hit rate, resulting in decreased server usage and increased performance for our users.
We’re Hiring!
If these kinds of projects and challenges sound interesting to you, Ibotta is hiring! Check out our jobs page for more information.
