Cache and Kaniko

Yannig Perre
Jul 13 · 9 min read
Image for post
Image for post
Artichoke flower

Foreword

This article is the result of various experiments that I have been able to carry out with Kaniko in order to minimize compilation times and the consumption of system resources.

Thereafter, the reader will have to have some knowledge on the operation of Docker and the creation of images. Docker installation will also not be discussed.

Building a program is a complicated process that requires a lot of action:
- Installation of specific tools (Maven, Yarn, Node etc)
- Source code recovery (Git, Subversion etc)
- Creation of construction script
- Archiving of deliverables in a tool (Artifactory, Nexus etc)

Thus, a simple program based on Spring boot will require Maven, a Java virtual machine, a Nexus server etc. Each element requires each particular care: installation of update (security or evolution), maintenance, backup etc.

From this point of view, Docker has made great changes in this area. The installation of a Docker server result in the following gains:
- Removal of the installation of construction tools (Maven, Java, Yarn, Node etc)
- Simplification of update processes
- Use of a single deliverable whatever the technology
- Simplification of storage tools for deliverables and replacement with a Docker registry

Image for post
Image for post
Strawberry and strawberry flower

Construction of a deliverable with Maven

Spring boot provides a git repository containing a Spring boot example. This small project is available at the following address: https://github.com/spring-guides/gs-spring-boot

Below is the command to clone it:

As mentioned before, this project requires a JDK and a Maven compiler. Under Ubuntu 20.04, this installation can be done using the following command:

Now go to the project directory in the complete sub-directory:

The construction of the deliverable is done using the mvn command followed by the clean package options. Precede this command by time in order to know the time required for compilation.

Below the corresponding command:

On my laptop, the construction of the deliverable takes about 45 seconds the first time (I have a connection to the fiber). We also see a lot of lines of the following form:

The second launch will see these lines completely disappear and the time required to build the deliverable will drop to 27 seconds, a gain of 18 seconds linked to the presence of the components already in the cache. These dependencies are downloaded once and then stored in a cache directory (by default ~/.m2/repository).

Project launch

Once compiled, the program is available as a JAR file in the target directory (under the name of target/spring-boot-0.0.1-SNAPSHOT.jar).

Below the java command you can use to launch this application:

Upon startup, the program should return a number of messages. Below is an excerpt from the output:

The application is started and available on port 8080 of the machine (http://localhost:8080). Call this address using the following curl command:

The command should return the following message:

Image for post
Image for post
Black tomatoes

Compiling using Docker

At first, this jar file will be integrated into a Docker image. Below is a minimal Dockerfile that could be used:

Drop this file in the gs-spring-boot directory.

The construction of the image is started using the docker command and the following options:
- The build instruction
- The -t option followed by the image name (-t gs-spring-boot for example)
- Finally, the path where the Dockerfile is located

To start building the image from the gs-spring-boot directory, the command to launch will be as follows:

If all goes well, the command should return the following result:

Now launch this image using the docker command and the following options:
- The instruction run
- The -it option to assign a virtual terminal to the container
- The --rm option to delete the container at the end of its instruction
- Finally the name of the image to launch (gs-spring-boot)

Below the complete order:

The program then launches in the same way as before.

Construction using Docker

The application is now in a container. However, the compilation of the program must always be done on my laptop with a Java virtual machine and a Maven interpreter.

In order to avoid installing all these elements, it is possible to use a Docker image in two phases:
- A first phase in which the deliverable is constructed
- A second phase taking the result of the first phase to make it available

Below is an example of an image allowing this compilation:

Save this definition as Dockerfile and start the construction with the following command:

The construction takes about 2 minutes (having previously recovered the basic images) due to the download of the Maven dependencies. The Maven build alone takes about 1 minute.

Test the image is done using the following command:

In terms of startup, nothing special to report, everything works as before.

The advantage is that from now on, it is no longer necessary to install Java and Maven on your computer.

Construction with Kaniko

The image is compiled and it starts correctly. Unfortunately, in a Kubernetes cluster, there’s several drawback to directly compile an image with Docker:
- Exposing control of the Docker daemon is strongly discouraged for security reasons in a Kubernetes cluster
- Kubernetes does not necessarily run with a Docker daemon

The preferred solution is to go through Kaniko.

Before integrating the compilation into a Kubernetes cluster, it is possible to use Kaniko in a Docker daemon for debugging. Generally, the trick will be to mount a volume (mounted in the /workspace directory) containing the sources to build and then start the construction. In order not to deposit the image on a register, the user will use the Kaniko — no-push option.

Below is a compilation command that can be used from the directory containing the Dockerfile:

The compilation is going well, nothing to report on this side here. However, at each launch, Kaniko is obliged to download all the images. The situation is therefore even worse compared to Docker since the latter had a cache of downloaded images.

Without cache, the construction takes 4 to 5 minutes with a strong connection. You need to download both images as well as Java dependencies with Maven. The Maven construction indicates a time of approximately 1 minute.

Kaniko’s execution time is significantly longer than that of Docker. This difference is explained quite simply when we observe the operating principle of Kaniko.

In the case of Docker, the launch of a compilation operation is based on the creation of a container. If the image is already there, this operation is almost done instantly. Once the operation ends, a snapshot is taken of the state of the container and this layer is used to constitute the final image of the container.

In the case of Kaniko, everything actually happens in the container file system. When executing a command, Kaniko proceeds as follows:
- Image recovery
- Decompression of the image in the container FS
- Launch of the command
- Taking a photo showing the differences between “before” and “after”

A good way to reduce these preparation times is to set up a cache of the images used.

So that this cache can persist from one launch to another, the /tmp/cache directory will be used and mounted in the /cache directory (default directory).

Only limitation: Kaniko is not able to manage this cache during execution. The cache preparation must be done upstream and is supported by the image warmer. Below the command allowing to carry out this operation:

NB: the warmer executable is available in the /kaniko directory of the basic Kaniko image. It is thus possible to use it in a CI/CD system (like Gitlab for example).

When the command is finished, the local /tmp/cache directory should contain image archives. Below is an example of files:

The cache is ready in the /tmp/cache directory. It remains to use it in the Kaniko container by mounting it in the /cache directory. This time, the command takes the following form:

At the construction level, nothing to report except the presence of the following messages indicating that there is a cache on Docker images:

This time, the compilation ends in about 2 minutes with always 1 minute dedicated to the Maven part. So we fall back on times similar to those of Docker.

Image for post
Image for post
Apple

The Maven cache is by default present in the /root/.m2 directory. In order to have a persistent cache of dependencies from one execution to another, this directory will be mounted as a volume from the /tmp/maven directory.

Below is the complete command to use:

The first compilation takes about 2 minutes. However, the following compilations only take 35 seconds due to the presence of the dependency cache.

This significant improvement is perfectly understandable because the dependencies of Maven are no longer part of the files observed by Kaniko. Another aspect, these files are reused from one execution to another avoiding having to download them with each execution thus preserving bandwidth consumption.

Image for post
Image for post
Plow

Conclusion

At first glance, without optimization, Kaniko takes a significantly longer time to perform the same operations compared to Docker. On the other hand, as soon as the image cache mechanism is put in place, the times return to comparable values. The last point which is particularly interesting being to be able to also set up a cache specific to Maven where Docker cannot support it (at least for the moment). The same type of mechanism can quite be implemented in the case of compilation based on npm/yarn.

I hope you were interested in the article. I will try to talk about an example of implementation in Tekton as soon as I have a little time.

The Startup

Medium's largest active publication, followed by +732K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store