Next.js + Netlify: File Naming and Caching Strategies

This post is part of a series on Next.js and Netlify. You can read the first part here.

The problem with Next.js’s default File Naming Strategy:

Next.js by default creates a 21 character random string for every build using nanoid. This is helpful in a way that all your assets will have the same prefix across one deployment.

Next.js stores the value of buildId in the BUILD_ID file in the /.next folder to determine which asset files to serve.

What this strategy also implies is that every asset in every deployment will have its own unique URL and new URLs will be created with every deployment. Now because no content of a file will be overwritten by any future deployment, we can assume that the contents of the file to be immutable and can be cached indefinitely by CDN and browsers. You can tell browsers that a URL’s content is immutable by serving a high HTTP max-age header. The browser will not even make a request for such an asset if it has one in its cache. It can help reduce cache invalidation woes on the browser as well because the content is immutable, you never need to invalidate the cache, browsers will automatically dispose of least accessed assets.

You can read more on Caching best practices and max-age gotchas from this article by Jake Archibald.

Caching best practices & max-age gotchas

But this means that on every deploy you will changing every file (including HTML files as they reference JS file names). This might result in longer deploy times and you would be bursting cache on your user’s browsers even if the file content has not changed. Thus Netlify recommends against high max-age cache.

Netlify Headers Cache Control For Static Assets

You can reuse the same file name and get better results through Netlify’s edge CDN. Netlify sets the headers for an asset something like this:

cache-control: public, max-age=0, must-revalidate

These headers tells the browser that it can cache the content indefinitely but every time before using it, it has with to check with the server once with the etag reference. The browser will make a request with this etag in the request headers. If the content has not changed the server will respond with 304 status code. And these requests are very fast when served by a CDN.

Read more about Netlify’s caching here:

Better Living Through Caching | Netlify

So before we adopt the default strategy, let’s look at a few other strategies:

Same file names across builds / deploys:

Now we don’t want the URLs to change every-time. What if we set the buildId a constant value? This seems like a good strategy if you are don’t add another layer of caching anywhere which needs to be invalidated. i.e. the only caching is done by Netlify and your user’s browser’s inbuilt caching mechanism. For every asset that is requested by your page, the browser will make a request to the server to validate what it has in cache is still fresh and applicable. You can achieve this by setting the buildId to be constant. (Through generateBuildId)

But, and there is a big issue, which to understand we need to look at how the router currently works:

Next.js Router Explanation. Credit Tim Neutkens. (From here.)

Next.js determines the path of the asset of a route using this:

`/_next/static/${buildId}/${route}.js

What will happen if a user has your website open in a browser tab while you push out a new deployment? If he clicks on a link, Next.js will request the asset which, because the buildId is constant, will be from the new deployment and try to use it to render the new route. This may or may not work depending on what you changed in your new deployment. If a vendor dependency is added to the commons bundle and this new route’s asset requires it will throw an error because we don’t reload the common vendor bundle.

So if you are going to have more than one page with linking to one another with client-side routing, don’t use this strategy with Next.js. You can still use this strategy with other frameworks which don’t have such client side routing when deploying to Netlify. Or disable client-side routing and make the server respond to all routes.

Content md5/sha hash incorporated in file name:

Here every built asset had it’s hash saved in it’s filename. Any modifications that you do to the source code modifies the filename of the asset where it finally gets bundled. If let’s say you modify the source code of a file which affects just one route and you have ensured that file is not imported in your _app.js then and only then that Page’s asset path gets modified and the other pages' asset path remain the same. Essentially every hash will point to one version of the code and can be considered immutable. So now you can use long term caching and the browser will never have to request for the same asset twice.

So now instead of using Next.js’s default strategy of changing all paths every time and changing no paths ever, this gives a perfect middle ground where the path of an asset changes only when its contents get changed.

The way to achieve this is not easy and does not have the blessings of Next.js developers so expect this to break in future versions of Next.js. But nevertheless with version 8.0.3 to achieve this you need to do the following:

1. Modify the filename function in the Webpack config to create client-side assets with content hash in the filename. (See here.)
2. You would need to save these content hashes somewhere so create a Webpack plugin which creates a JSON file which stores the mapping from route to content hash i.e. the asset path. (See here.)
3. Use this JSON at build time to create <script> tags in the Server Side Rendering part (i.e. in pages/_document.js in the NextScript Component) (See here.)
4. When the user clicks on a link, your client-side router needs to request the correct asset path mapped to the route that the user is requesting to go to. This involves Monkey patching the PageLoader to generate assets paths using the mapping that you build in step 2. You will be able to do this because Next.js currently does not mangle props. (See here.)
5. You need to make sure that you serve old and new assets with Netlify. i.e. Retain files across builds along with retaining Webpack’s records.json. (Coming up soon in a later blog post)

You can take a look at this PR which covers all of the above.

(Side note: this is what I love and hate about Javascript right now, I can modify a class’s method from outside the library. This will not be possible when private fields and methods proposal will come out and libraries start using it.)

Conclusion

Subscribe to Github Issues to be updated on solutions:

• zeit/next.js #6303 use content hash in pages chunk name
• zeit/next.js #7329 [RFC] useLink

If you are developing for a battle-tested, production-ready service for your company or for a client where you don’t want to leave any technical debts, go with Next.js default strategy.

If you are building your own portfolio site (me 😅) for which you can afford to experiment with and want the best performance with Next.js + Netlify, use the content hash strategy. Also don’t forget to leave a link to this blog post in your source code for your future self to reference.

Update: 15th May 2019: There is an RFC to Next.js which would not require us to implement the solution present in this blog. Subscribe here.

Update 10th August 2019: The above mentioned RFC has been dropped as returning a component from a hook will have caching issues.

What’s next:

So these are a couple of blogs that I intend to write on Next.js + Netlify.

1. Introduction
2. File Naming and Caching Strategies (this one)
3. … (more coming soon, when I get time to write them 😅)

I’m writing these posts as I learn while building my blog. Do follow me to get notified when they are ready. Do recommend (👏 ) this if this article helped you get started with Next.js or helped improve your Next.js setup.