An HTTP Caching Strategy for Static Assets: The Problem

This post is the first in a four-part series.

In Adobe Experience Platform Launch, we provide a web application where our users manage their marketing technologies. To power the user interface, a number of assets (HTML, JavaScript, and CSS files, for example) must be shipped from our servers to the user’s browser.

Caching, in this context, is the technique of storing a copy of these assets on the user’s machine so when they use our application again or navigate to different pages that use the same files, the user’s browser doesn’t have to load the files from our servers again. Instead, the browser can re-use the copy of the assets that are on the user’s machine, which is a much faster process and results in a snappier user experience. As a side benefit, it also reduces the load on our servers.

We don’t always want the browser to use the files from cache, though. If we add a feature or fix a bug and publish the change to our servers, we want our users to be able to get the updated assets immediately rather than using the outdated assets.

How and when to cache is a balancing act and one that can lead to slow or buggy experiences if not designed or implemented correctly. In the beginning phases of building Launch, we noticed that after we would release a user interface update, odd things would sometimes happen; the application wouldn’t load or certain views would fail. After a hard refresh in the browser (clearing the browser cache), things would begin to work normally again. It became clear that this was due to a lack of a caching strategy — something we knew we needed but had not been our highest priority.

Default caching behavior

Not having a caching strategy doesn’t mean caching isn’t happening. Quite the opposite, in fact. The HTTP/1.1 spec states:

Since origin servers do not always provide explicit expiration times, HTTP caches typically assign heuristic expiration times, employing algorithms that use other header values (such as the Last-Modified time) to estimate a plausible expiration time. The HTTP/1.1 specification does not provide specific algorithms, but does impose worst-case constraints on their results. Since heuristic expiration times might compromise semantic transparency, they ought to [be] used cautiously, and we encourage origin servers to provide explicit expiration times as much as possible.

In simpler terms, if a server doesn’t indicate if or how a file should be cached, browsers will typically attempt to determine a reasonable amount of time to cache the file. The spec goes on to state:

Also, if the response does have a Last-Modified time, the heuristic expiration value SHOULD be no more than some fraction of the interval since that time. A typical setting of this fraction might be 10%.

When a server transfers a file to the browser, it usually sends a timestamp of when the file was last modified. Here, the spec is saying that if a file gets sent and a timestamp is included, then the browser should cache the file for less time than the difference between when the file was transferred and the “last modified” timestamp. For example, if we’re requesting a file at the end of January and the file was last modified at the beginning of January, then the browser shouldn’t try to cache the file for any longer than one month. It also states that a typical approach would be to cache the file for 10% of this amount of time. In our example, this means the file would be cached for around 3 days (31 days / 10).

So what happens in reality? Browsers follow what the spec says, actually. Here’s the equation in the Chromium (Chrome) source code, Gecko (Firefox) source code, and WebKit (Safari) source code.