How to speed up Angular build times with caching

TL;DR We were able to reduce build times to 6 minutes (from 17) and test times to 18 seconds (from 1–2 minutes).

At Vendasta, we have many Angular repositories, some large and some small. Over time, our CI/CI pipeline in one of our larger pipelines has slowed down. We were reaching build times upwards of twenty minutes, which was the hard limit of time we added as a safety net to our CI system — so they could have been longer!

20 minutes might seem short for some companies’ build times, but at Vendasta, instead of large releases on a multi-day/week/month cadence, we are shipping small amounts of code to production hundreds of times per day. Even while working on your branch, each commit you make would run a build that would take just as long.

We needed to speed up build times to enable developers to move faster and allow developers to be confident in the build process. It also wouldn’t hurt to save money in Cloud Build charges by reducing build times.

Our current CI/CD Pipeline

Google Cloud Build (https://cloud.google.com/cloud-build)

We use Google Cloud Build for our CI/CD pipeline, which works great for both our frontend and backend builds since you can specify any configuration. The only problem for our frontend builds is Cloud Build is inherently stateless, meaning from build to build nothing is carried over. That means any of the caching of the Angular build process or Jest test caching is lost between builds.

When building locally, I noticed the build times after running the first build was dramatically faster. I searched around the web to find out what is happening and found almost no results. The documentation for ng build also does not talk about the caching strategy, so I started my investigation.

Searching for some cache 💸

I noticed when you run the ng build command, after everything is finished running a new directory is created in your node_modules folder called .cache with a few subfolders.

.cache directory after running the build

I tried removing that .cache folder then rebuilding the application and, voilà, slow build times are back. Now we know where the cache exists but how do we get that working in Cloud Build since a new `node_modules` is created for every build.

It’s dangerous to go alone! Take node_modules

In our current Cloud Build yaml, we run npm ci every time we build our projects. If you do not know what npm ci does, this is a snippet from the docs:

This command is similar to npm-install, except it’s meant to be used in automated environments such as test platforms, continuous integration, and deployment – or any situation where you want to make sure you’re doing a clean install of your dependencies.

This command is exactly what we want to use in CI but this command also completely removes the previous node_modules folder which would defeat the purpose of the caching anything within the node_modules . This is what our current yaml looks like (some steps have been removed):

We decided to cache the entire node_modules folder. When we move to Angular 9 this would also allow us to cache the ngcc compilations. (for more information on that https://angular.io/guide/ivy#ivy-and-libraries). Caching the node_modules folder is usually a bad idea as people running different node versions or different environments can cause problems, but since this is in our CI/CD where we can control each step and version we are using, we were fine with this risk.

Before we can cache node_modules, we need a way to identify when we should be using the cache or doing a npm ci . The way we created that check was hashing the package-lock.json file since that should only be updated when there are new dependencies. This means that if a new dependency is added or the package-lock is updated in any way, it will not use the cached node_modules which we found is fine.

Getting node_modules from cache, heavily commented

As you can see from the above code, we check with the cache first and if we fail to find a cached node_modules we npm ci and that is it! Simple as that.

Put all the cache in the bag!

After building the application, we need a way to store the new, updated cache. Storing the cache is a simple process using Google Cloud Storage:

Storing the cache in GCS, heavily commented

As you can see from the above, we only store the cache if we didn’t find a previous cache. This is perfect for non-master builds but on master you most likely want to update the cache every time. So for master remove the line test ! -f node_modules.tar.gz and it will overwrite the cache.

A little bonus time gain 🎉

If you use Jest or any other test runner for your frontend, you could potentially benefit from also saving the test cache. We use Jest for majority of our frontends, so I am going to only explain how to cache with Jest.

If you add “cacheDirectory”: “node_modules/.cache/jest” to your Jest config, it will store any test cache in the same place Angular stores its cache. With Jest cached, we have seen some tests go from 60 seconds (!!!) down to 3 seconds.

Some gotchas

Make sure your GCS bucket does not have a retention policy, as that will not let you overwrite your cache. You can set the bucket to automatically delete old cache versions after X amount of time to keep costs down.

Graham Holtslander was a huge help in researching and implementing this solution. Thanks Graham!
Also thank you to
Shane Williamson and Ben Stolz for reviewing the draft of this post.

Vendasta

Development stories and insights from the leading end-to-end white-label B2B platform, used by agencies and media companies who sell to local business

Thanks to Corey Hickson

Mitchell Woodhouse Mckenzie

Written by

Frontend Infrastructure Developer at Vendasta

Vendasta

Vendasta

Development stories and insights from the leading end-to-end white-label B2B platform, used by agencies and media companies who sell to local business

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade