Creating Your Own Apollo Cache for Perf and Profit

Optimizing page loading time with a custom Apollo cache

Photo by Sanwal Deen on Unsplash

In the process of performance tuning any React Apollo app, it may become necessary to take a closer look at your caching strategy. Most of the current advice on optimizing caching performance of Apollo entails minimizing the number of network calls required to fetch all of the data for your application’s queries. These include important techniques such as cache redirects, prefetching, and query splitting. Once you’ve exhausted these techniques, you may find the need to write your own custom caching solution that’s tailored to your application’s needs. Unfortunately, the Apollo docs don’t currently cover how to write a cache of your own, so in this post I hope to shed some light on Apollo’s caching API and make it easier for you to write a cache that suites your needs.

Motivation

I was recently looking to improve the performance of my React Apollo application that consists of a single page which calls one large GraphQL query on initial page load. This query is executed server-side for server-side rendering and is fairly large, with over 1000 fields. To figure out how expensive this query might be, I took a CPU snapshot of production traffic. I was surprised to determine that on average, our Node process was spending about 20% of its non-idle time writing GraphQL query results to the default InMemoryCache for the server render.

CPU profile of production traffic for our React GraphQL search application with the “InMemoryCache.write” call highlighted.

As the Apollo docs indicate, the default InMemoryCache works by breaking up a GraphQL query result into its constituent schema objects and stores it in a map. The key that it uses is either the object id (either in the schema or from dataIdFromObject) or the path to the object within the GraphQL query. By normalizing the result in this way, InMemoryCache can reduce the cache size for large amounts of repeated data. In addition, it allows the cache to return partial results for other queries, reducing the overall number of network calls required.

Unfortunately this normalization process, as we saw in our CPU snapshot, is CPU-intensive. This is due to the nature of how normalization works. It takes the GraphQL results, which are in the form of a tree, breaks the tree up into its constituent objects, and stores the objects individually in a map.

Writing and reading to the cache deconstructs and reconstructs the GraphQL result tree into its constituent objects.

For large queries, this process can eat up quite a bit of CPU as well as introduce a bit of latency (as must as 60–70ms in our case).

How can we make this better for our app?

Our app has one giant GraphQL query that is executed server-side and returned to the client using the SSR Apollo recipe. We mostly care about getting the page to the user as fast as possible, so it doesn’t make much sense to do this extra work when all we want is the results on the page.

Ideally, what we want is to serialize the GraphQL results and use them directly in our page, bypassing the normalization process. However, for client side queries, we’d still want to use the InMemoryCache because we’re not as sensitive to CPU usage or latency there. What we need here is a slightly customized InMemoryCache with a special case for the initial server side rendered query.

With a high level idea of the caching we require, we now need to look into the Apollo caching API itself to understand how to implement our caching strategy.

Looking Under the Hood of InMemoryCache

When looking at the Apollo caching API, there are just 2 main classes to look at: the ApolloCache base abstract cache, and the InMemoryCache which inherits from it. In the base class, we see that the core caching API can be summarized with the following interface:

The types and interfaces of the API arguments and return values are as follows:

Core ApolloCache API

Looking more closely at the ApolloCache interface, we see that the core API consists of

• read: This is the main read method which readQuery and readFragment use to fetch the cached results of a given GraphQL query or fragment. readQuery and readFragment are already implemented on the ApolloCache base class, so all that’s required is a read implementation.
• write: The main write’ method which writeQuery and writeFragment use to write the results of a GraphQL query or fragment to the store. Just like read, ApolloCache provides implementations for writeQuery and writeFragment so all that’s needed is a write implementation.
• diff: Method used by the ApolloClient to return as many cached fields for a given GraphQL query as possible. It returns a flag indicating whether or not all the query’s fields were present in the cache. If any are missing, the ApolloClient will need to fetch the additional fields from the server.
• watch: Takes a callback which is executed whenever any data for a given GraphQL query is updated in the cache. If an unrelated query returns updated fields for the same object as a previous query this allows the UI to re-render with the new data without making an explicit call to the client.
• reset: Clears the cache contents.

Server-side rendering/offline storage API

If we would like to support server-side rendering, we need to implement:

• extract: Returns a serialized version of the ApolloCache which will be passed to restore client-side. The exact serialized format doesn’t matter so long as restore properly de-serializes it.
• restore: Takes the output of extract and uses it to populate the cache with the given serialized data.

Transaction API

Lastly, if our app is performing mutations, we will need to implement the transactional API used for supporting the optimistic UI feature of Apollo

• recordOptimisticTransaction: Takes a transaction to do an ‘optimistic’ update of the cache prior to performing the mutation on the server. Internally, InMemoryCache implements this by creating a copy of the cache contents which will contain the optimistic updates. These can then either be ‘committed’ by calling performTransaction or rolled back via removeOptimistic
• removeOptimistic: Rolls back an optimistic transaction with the given ID.
• performTransaction: Takes a transaction to perform on a cache. This method is used to commit the results of a successful mutation after the data is updated on the server.

Initial Page Load Optimized Cache

Now that we understand how the ApolloCache API works, we can turn to implementing a cache which optimizes initial page load performance.

Since we only want to customize the behavior for the initial server side query, it is sufficient to just subclass InMemoryCache and override the methods to inject our custom behavior while delegating to the InMemoryCache implementation for all other queries.

Implement it!

Referring to the cache API above, we see that we will need to override all of the core API methods as well as extract and restore. I skipped overriding the transactional API since my application is a read-only app with no mutations. If you need the transactional API, it is straightforward to extend my implementation to take into account optimistic changes by referring to InMemoryCache's implementation.

Sketching out the customized behavior we need to have:

• constructor: Pass the named query‘s name in the initialQueryName constructor argument and store it on the instance
• write: If initialQueryName is specified, we need to cache the results of the first invocation of initialQueryName into a variable _INITIAL_QUERY; otherwise, default to the InMemoryCache implementation
• read: If _INITIAL_QUERY is present and its variables match the incoming request, return it; otherwise default to InMemoryCache
• diff: Since we are not caching individual fields, this implementation is nearly identical to read, with the addition of returning the necessary complete: true field.
• extract: Merge _INITIAL_QUERY with the result of InMemoryCache#extract and return it
• restore: Restore _INITIAL_QUERY and pass the rest to InMemoryCache#restore.
• reset: Clear _INITIAL_QUERY and call InMemoryCache#reset

Implementing the above we arrive at

How did it perform?

When I load tested the above implementation in our search application and compared it with the non-customized version, I measured a 7% reduction in CPU usage and a 7.2% reduction in memory usage. Given how small this tweak was and how much we leveraged the existing implementation in InMemoryCache, these are some pretty good savings.

CPU usage, top curve is InMemoryCache, lower curve is OptimizedInMemoryCache

Memory usage in MB, top curve is InMemoryCache, lower curve is OptimizedInMemoryCache

Closing Remarks

If you got this far, I hope you came away with a better understanding of how caching works in Apollo and that you are inspired to dig deeper. Please feel free to leave a comment or question.