Fasten Docker build

Published in

Opstree

5 min readMay 12, 2020

Context

Recently I started working on a microservices project, as a DevOps engineer my responsibility was to ensure smooth build and release of the project. One of the challenges that I was facing was the image building process of the projects was painfully slow. Following true Opstree spirit of continuous improvement I started exploring how I can fix this problem and finally got a decent success, I was able to reduce docker image build time from 4 minutes to 20 seconds. In this blog, I would like to showcase various ways through which image building can be reduced drastically.

You can find the complete code available here in this repository

Problem statement

I’m using a Springboot HelloWorld project to walk you through the problem statement and eventually the various solutions that we would be applying.

The base of the problem lies in the fact that 80–90% of image building time is consumed in downloading the dependencies defined in pom.xml. Since the scope of downloaded dependencies is limited to the image build process only, that’s why every time an image builds the complete process starts from the beginning.

FROM maven:3-jdk-8
 
LABEL maintainer="opensource@opstree.com"
 
WORKDIR /usr/src/app
 
ADD . /usr/src/app
RUN mvn clean package -Dmaven.test.skip=true
 
ENTRYPOINT ["java","-jar","/usr/src/app/target/helloworld-0.0.1-SNAPSHOT.jar"]

If we build a docker image using the above Dockerfile, it will take close to 3 minutes to build the image.

$ make problem-build-package-with-time 
real	3m3.356s 
user	0m1.221s 
sys	0m1.067s

Solution1 | Avoid downloading dependencies

We know the solution to this problem lies in the fact “ if somehow we can skip downloading dependencies “ our problem will be solved. If there would have been an option to mount the host system local repository( ~/.m2) while building the image this problem would have been resolved.

We have solved this problem by moving the artifact generation part out of the image building process. Artifact generation is done using a build container having the local maven repo mounted so that dependencies would be downloaded only if not already present.

solution1-build:
    docker run -it -v ~/.m2/repository:/root/.m2/repository \
     -w /usr/src/mymaven -v ${PWD}:/usr/src/mymaven --rm \
     maven:3-jdk-8 mvn clean package -Dmaven.test.skip=true

Once the artifact generation is done the only thing that you have to do in your Dockerfile is to copy the generated artifact in your Docker image.

FROM maven:3-jdk-8 
LABEL maintainer="opensource@opstree.com" 
WORKDIR /usr/src/app 
ADD target/helloworld-0.0.3-SNAPSHOT.jar /usr/src/app/app.jar ENTRYPOINT ["java","-jar","/usr/src/app/app.jar"]

Now if we build our image post this solution the image build time will drastically reduce to ~20 seconds

solution1-build: 
docker run -it -v ~/.m2/repository:/root/.m2/repository \ 
-w /usr/src/mymaven -v ${PWD}:/usr/src/mymaven --rm \ 
maven:3-jdk-8 mvn clean package -Dmaven.test.skip=true solution1-package:
 docker build -t opstree/fasten-build -f Dockerfile.solution1 . solution1-build-package:
 make solution1-build 
make solution1-package solution1-build-package-with-time: 
time make solution1-build-package >/dev/null 2>&1 $ make solution1-build-package-with-time time make solution1-build-package >/dev/null 2>&1
 real	0m15.212s 
user	0m0.288s 
sys	0m0.286s

Solution2 | Leverage docker layer caching

One of the very interesting concepts of docker is the caching of layers where a layer is only built if there are supposed to be some changes.

FROM maven:3-jdk-8 LABEL maintainer="opensource@opstree.com"WORKDIR /usr/src/app ADD pom.xml /usr/src/app
 RUN mvn dependency:resolve -Dmaven.test.skip=true ADD . /usr/src/app 
RUN mvn clean install -Dmaven.test.skip=trueENTRYPOINT ["java","-jar","/usr/src/app/target/helloworld-0.0.3-SNAPSHOT.jar"]

If you notice lines 7 & 8 are the new addition in comparison to the original Dockerfile. When the image will be built using this Dockerfile, the layers corresponding to lines 7 & 8 will be only built if & only if there is a change in pom.xml else the previously built layers cache will be used. Hence no time will be wasted in downloading the compile-time dependencies(A concept unique to maven).

solution2-build-package:
 time docker build -t opstree/fasten-build -f \ 
Dockerfile.solution2 . solution2-build-package-with-time: 
 time make solution2-build-package >/dev/null 2>&1 $ make solution2-build-package-with-time 
time make solution2-build-package >/dev/null 2>&1 real	1m32.215s 
user	0m0.788s 
sys	0m0.787s

Solution 3 | Best of both worlds

Solution2 worked pretty well, the only problem in the approach was that if even there would be 1 single change of line in pom.xml the dependencies layer would be built again.

FROM maven:3-jdk-8 LABEL maintainer="opensource@opstree.com" WORKDIR /usr/src/app ADD . /usr/src/app 
RUN mvn clean package -Dmaven.test.skip=true

In solution 3 we are sort of combining Solution1 and Solution 2 where the dependencies downloading is moved out via an intermediate builder image as shown above.

The final image build process will use this build image as a base image that will already have all the dependencies inside it. Even if there is a new dependency added in pom.xml the final image will only download that delta dependency as the rest of the dependencies would already be present via base builder image.

FROM opstree/fasten-build-builder ADD . /usr/src/appRUN mvn clean package -Dmaven.test.skip=trueENTRYPOINT ["java","-jar","/usr/src/app/target/helloworld-0.0.3-SNAPSHOT.jar"]

The complete image building process would be executed as given below, please note that you don’t have to build the builder image frequently it can be a scheduled operation nightly or weekly depending on the dependencies update in your pom.xml.

solution3-build-builder:  
  docker build -t opstree/fasten-build-builder \ 
  -f Dockerfile.solution3.builder .solution3-build-package: 
  docker build -t opstree/fasten-build \ 
-f Dockerfile.solution3 . solution3-build-package-with-time: 
  time make solution3-build-package >/dev/null 2>&1 
  
$ make solution3-build-package-with-time 
 time make solution3-build-package >/dev/null 2>&1 real	0m4.800s user	0m0.224s sys	0m0.200s

Conclusion

In conclusion, I would like to summarise that you can either go for Solution 1/2 or 3 on case to case basis. Solution 2 will be apt when your pom.xml is stabilized and there are rarely any changes in pom.xml.

Also, you would have noticed that the Dockerfile is not written as per best practices i.e non-root user, multistage docker build that I didn’t cover intentionally. That I would like to cover in the next blog.

More importantly, I would like to stress the most important part that is continuous improvement whenever you do work it’s fine that you start with a workable solution but then you should be in constant pursuit of taking it to next level.

Feel free to give any feedback or recommendation would love to hear your thoughts or if you have another smart solution that would be great. Till we meet next time Happy Learning.

Gif reference: https://giphy.com/gifs/the-flash-Z2pZfL0YPC9sk

Originally published at http://blog.opstree.com on May 12, 2020.