How to build Nodejs Docker image using cache

If you’re using Docker images for your CI/CD processes you’ve most likely noticed one thing — npm install stage is constantly executing even when npm dependencies list has not changed 😐

That’s because Docker invalidates cache every time your repository changed (so the image layer is always a bit different). And in the common cases, you can see that it’s downloading all the npm packages again.

The standard way

Let’s got through some examples, this is the case when Docker downloads all npm packages every time you push or save new changes to the application repository.

FROM node:10.14.0
WORKDIR /usr/src/app
COPY . .
RUN npm install
CMD ['npm', 'start']

Put this file to the directory along with some package.json , package-lock.json and arbitrary file of your choice.

Now try to build this image with docker build command.

time sudo docker build -t bla .
Sending build context to Docker daemon  18.94kB
Step 1/5 : FROM node:10.14.0
 ---> d330537fea0f
Step 2/5 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 4a7939a1a682
Step 3/5 : COPY . .
 ---> Using cache
 ---> 21530de495a6
Step 4/5 : RUN npm install
 ---> Running in d5dd9ac1a0c2
npm WARN node-js-sample@0.2.0 No repository field.
added 48 packages from 36 contributors and audited 121 packages in 1.327s
found 0 vulnerabilities
Removing intermediate container d5dd9ac1a0c2
 ---> fd0c61d2caf1
Step 5/5 : CMD ['npm', 'start']
 ---> Running in d3f81b283eff
Removing intermediate container d3f81b283eff
 ---> 82c7f839044f
Successfully built 82c7f839044f
Successfully tagged bla:latest
real 0m3,360s
user 0m0,064s
sys 0m0,015s

Now try to trigger the same build command again.

time sudo docker build -t bla .
Sending build context to Docker daemon  18.94kB
Step 1/5 : FROM node:10.14.0
 ---> d330537fea0f
Step 2/5 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 4a7939a1a682
Step 3/5 : COPY . .
 ---> Using cache
 ---> 21530de495a6
Step 4/5 : RUN npm install
 ---> Using cache
 ---> fd0c61d2caf1
Step 5/5 : CMD ['npm', 'start']
 ---> Using cache
 ---> 82c7f839044f
Successfully built 82c7f839044f
Successfully tagged bla:latest
real 0m0,203s
user 0m0,114s
sys 0m0,023s

See? It has been done in less than a second! But everything changes when you change the content of your file. This simulates situation that happens in every repository every single day — developers make changes, that’s why you hired them for 😀

echo "console.log('it works.');" >> index.js
time sudo docker build -t bla .
Sending build context to Docker daemon  18.94kB
Step 1/5 : FROM node:10.14.0
 ---> d330537fea0f
Step 2/5 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 4a7939a1a682
Step 3/5 : COPY . .
 ---> d0c72d36e13f
Step 4/5 : RUN npm install
 ---> Running in 55913c735508
npm WARN node-js-sample@0.2.0 No repository field.
added 48 packages from 36 contributors and audited 121 packages in 1.26s
found 0 vulnerabilities
Removing intermediate container 55913c735508
 ---> ab8fe3798e78
Step 5/5 : CMD ['npm', 'start']
 ---> Running in 98c0d79bc964
Removing intermediate container 98c0d79bc964
 ---> 2636693b69d1
Successfully built 2636693b69d1
Successfully tagged bla:latest
real 0m3,638s
user 0m0,072s
sys 0m0,003s

Huh, it was downloading all the npm packages again event though we did not change contents of package.json . As I’ve already said — that’s because the stage has changed (minimal change is still the change) so Docker invalidated the cache.

I have only one package in the dependencies so that’s why it takes only 3 seconds 😉 However in the real life scenarios this could go much worse.

Separate static from the dynamic stuff

But we know that at least two things in the repository don’t change so often — package.json and package-lock.json . So we can just put these files to the different stage and execute npm install only with them, right? This is the idea:

FROM node:10.14.0 as builder
WORKDIR /usr/src/app
COPY package.json .
COPY package-lock.json* .
RUN npm install
FROM node:10.14.0
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/ /usr/src/app/
COPY . .
CMD ['npm', 'start']

Let’s build run the first build.

time sudo docker build -t bla .
Sending build context to Docker daemon  18.94kB
Step 1/10 : FROM node:10.14.0 as builder
 ---> d330537fea0f
Step 2/10 : WORKDIR /usr/src/app
 ---> Running in 70c45d10910e
Removing intermediate container 70c45d10910e
 ---> d0a409c277ef
Step 3/10 : COPY package.json .
 ---> 59378ff66e03
Step 4/10 : COPY package-lock.json* .
 ---> 09996668bc22
Step 5/10 : RUN npm install
 ---> Running in e83613ff351c
npm WARN node-js-sample@0.2.0 No repository field.
added 48 packages from 36 contributors and audited 121 packages in 1.25s
found 0 vulnerabilities
Removing intermediate container e83613ff351c
 ---> 2cc47cf11c34
Step 6/10 : FROM node:10.14.0
 ---> d330537fea0f
Step 7/10 : WORKDIR /usr/src/app
 ---> Using cache
 ---> d0a409c277ef
Step 8/10 : COPY --from=builder /usr/src/app/ /usr/src/app/
 ---> 5c8e9a59c1b7
Step 9/10 : COPY . .
 ---> 223f190c3c70
Step 10/10 : CMD ['npm', 'start']
 ---> Running in 941f90a6eb3d
Removing intermediate container 941f90a6eb3d
 ---> fbf896881a9c
Successfully built fbf896881a9c
Successfully tagged bla:latest
real 0m5,848s
user 0m0,066s
sys 0m0,015s

It took a few seconds but we were expecting that — it was the first build so cache just did not exist. Now we just need to apply the same method as before and build the image again.

time sudo docker build -t bla .
Sending build context to Docker daemon  18.94kB
Step 1/10 : FROM node:10.14.0 as builder
 ---> d330537fea0f
Step 2/10 : WORKDIR /usr/src/app
 ---> Using cache
 ---> d0a409c277ef
Step 3/10 : COPY package.json .
 ---> Using cache
 ---> 59378ff66e03
Step 4/10 : COPY package-lock.json* .
 ---> Using cache
 ---> 09996668bc22
Step 5/10 : RUN npm install
 ---> Using cache
 ---> 2cc47cf11c34
Step 6/10 : FROM node:10.14.0
 ---> d330537fea0f
Step 7/10 : WORKDIR /usr/src/app
 ---> Using cache
 ---> d0a409c277ef
Step 8/10 : COPY --from=builder /usr/src/app/ /usr/src/app/
 ---> Using cache
 ---> 5c8e9a59c1b7
Step 9/10 : COPY . .
 ---> ba432390e682
Step 10/10 : CMD ['npm', 'start']
 ---> Running in 5f640531448f
Removing intermediate container 5f640531448f
 ---> 76153021e44b
Successfully built 76153021e44b
Successfully tagged bla:latest
real 0m0,709s
user 0m0,068s
sys 0m0,012s

Yes 🎉 And this mechanism will work as long as package.json and package-lock.json are the same.

and believe me or not — everything’s where it’s meant to be:

sudo docker run -it --rm bla bash
root@13c672e08445:/usr/src/app# ls -lah
total 44K
drwxr-xr-x  1 root root 4.0K Dec  2 16:56 .
drwxr-xr-x  1 root root 4.0K Dec  2 16:55 ..
-rw-r--r--  1 root root   24 Dec  2 15:35 .dockerignore
-rw-r--r--  1 root root  230 Dec  2 16:55 dockerfile
-rw-r--r--  1 root root  208 Dec  2 16:56 index.js
drwxr-xr-x 51 root root 4.0K Dec  2 16:55 node_modules
-rw-r--r--  1 root root  13K Dec  2 15:39 package-lock.json
-rw-r--r--  1 root root  420 Dec  2 15:45 package.json

Wrapping Up

Today we’ve gone through the simple scenario which might accelerate some repeating build steps. This scenario won’t work for every Nodejs application, sometimes you just need to do all the stuff hard way and bypass useful Docker functionalities. But that’s the life we chose. There’s nothing to worry about 😂

Do you have any questions or ideas on how to improve this scenario? Do not hesitate to leave a comment!