Kaniko — Building Container Images in a Kubernetes Cluster
Introduction
How do you build container images locally? The answer is most likely that you are executing commands like:
$ docker image build [OPTIONS] PATH | URL | -But let’s move to the cluster and how do you build container images inside Kubernetes clusters as part of your continuous integration pipelines or any other process. Because remember we are moving most of the things to Kubernetes and that applies to building container images.
Within Kubernetes clusters Docker is a bad option because docker cannot run inside containers. Using Docker to build images within containers or inside Kubernetes clusters requires communication to the Docker engine running in cluster nodes. And that is a big security concern.
But if that security concern is not enough Dockershim, the one responsible for communication between Kubernetes and Docker is deprecated in Kubernetes v1.20. And sooner, depending on when you are reading this blog, it will be removed from Kubernetes. So, even if you are fine with running Docker inside of containers, even though that’s only a client that communicates with Docker engine running on the node and does the communication through socket and all that madness, even if you’re fine with all that which you shouldn’t be in any form or way, Docker will be removed, now that it is deprecated, from Kubernetes clusters. That means that you will be using alternative container engines now that there will be no Docker.
What are containers?
Before diving into Kaniko, it is crucial to briefly clarify what is a container in software industry.
In the application layer, containers are an abstraction where code and dependencies are built or packaged together. It is a piece of software that you may use to wrap your software process or microservice and make it runnable in any computing environment. Generally speaking, containers can be used to hold any type of executable file, including configuration files, libraries, code, and binary files.
On a single system, several containers may be running. Each container instance runs as a separate process while sharing the OS kernel with other containers. A sample application or microservice is deployed for use via the container platform after being packed into a container image.
Moreover, a container image is an application’s code, runtime, system tools, libraries and settings all packed up in a lightweight, independent executable package.
What is Kaniko?
Kaniko is a daemonless container image builder that allows users to build container images, inside a container or Kubernetes cluster. Since Kaniko is not dependent on Docker daemon it creates container images in Kubernetes cluster in userspace without using Docker Daemon. It was a project initiated by Google to remove the need for elevated privileges when performing container image builds.
Let’s explore Kaniko!
Why Kaniko?
So you should not be building container images on Docker inside Kubernetes clusters and even if you ignore me and you say hey you’re talking nonsense what will you do when Docker will disappear from Kubernetes. So, you must look for alternative method to build images as Docker will not an option anymore within Kubernetes.
So the alternatives. What are the alternatives?
There are more then 10 viable alternatives out there but today we will talk about the one that I like the most and that one is called Kaniko.
It is a tool to build container images from a Dockerfile just like Docker does but the major difference is that Kaniko can run inside containers and that means that it can run inside the Kubernetes clusters. There is no need for privileged mode and there is no need for exposing any sockets . There is no need for Docker running on nodes of our cluster as it does not really matter which container engine we are using for running containers. What does matter is that Kaniko can build container images inside containers and automatically inside Kubernetes clusters.
Now, let’s explore how Kaniko works by looking at a hands-on demonstration
How does Kaniko work?
Let’s dig into internals of Kaniko and explore about how does Kaniko launch containers
The filesystem from the base image is extracted to a local directory (e.g., /kaniko/imagename) by Kaniko once it has parsed the Dockerfile and retrieved the base image. The commands contained in the docker file are then translated into shell commands and used to build several stages. By carrying out these actions, Kaniko directly alters the local filesystem. This presents no problems because Kaniko is designed to run within an already-existing container, but if it were to run from within a host, things might not go well.
The procedure of taking a snapshot, compiling the change files into a tarball, and adding files to a layer map or whiteout then takes place. Kaniko uses a hashing algorithm that incorporates the file’s modification time (mtime) to detect whether the files have been modified.
Kaniko requires following instructions to build a Docker image:
- Path to Dockerfile
- Path to build context
- URL/Destination of the repository where the image will be pushed
Prerequisites:
- Kubernetes cluster
- Dockerhub account
- Config files (pod.yaml, volume.yaml, volume-claim.yaml)
Demo:
Step 1:
Prepare local mounted directory and create a simple Dockerfile in it
$ mkdir kaniko && cd kaniko
$ echo 'FROM ubuntu' >> dockerfile
$ echo 'ENTRYPOINT ["/bin/bash", "-c", "echo hello"]' >> dockerfile
$ cat dockerfile
FROM ubuntu
ENTRYPOINT ["/bin/bash", "-c", "echo hello"]
$ pwd
/home/<user-name>/kaniko # copy this path in volume.yaml fileStep 2:
To authenticate with Docker registry create a Secret named “regcred”
kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>Step 3:
Create resources in Kubernetes cluster:
# create persistent volume
$ kubectl create -f volume.yaml
persistentvolume/dockerfile created# create persistent volume claim
$ kubectl create -f volume-claim.yaml
persistentvolumeclaim/dockerfile-claim created# check whether the volume mounted correctly
$ kubectl get pv dockerfile
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
dockerfile 10Gi RWO Retain Bound default/dockerfile-claim local-storage 1m# create pod
$ kubectl create -f pod.yaml
pod/kaniko created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
kaniko 0/1 ContainerCreating 0 7s# check whether the build complete and show the build logs
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
kaniko 0/1 Completed 0 34s
$ kubectl logs kaniko
➜ kubectl logs kaniko
INFO[0000] Resolved base name ubuntu to ubuntu
INFO[0000] Resolved base name ubuntu to ubuntu
INFO[0000] Downloading base image ubuntu
INFO[0000] Error while retrieving image from cache: getting file info: stat /cache/sha256:1bbdea4846231d91cce6c7ff3907d26fca444fd6b7e3c282b90c7fe4251f9f86: no such file or directory
INFO[0000] Downloading base image ubuntu
INFO[0001] Built cross stage deps: map[]
INFO[0001] Downloading base image ubuntu
INFO[0001] Error while retrieving image from cache: getting file info: stat /cache/sha256:1bbdea4846231d91cce6c7ff3907d26fca444fd6b7e3c282b90c7fe4251f9f86: no such file or directory
INFO[0001] Downloading base image ubuntu
INFO[0001] Skipping unpacking as no commands require it.
INFO[0001] Taking snapshot of full filesystem...
INFO[0001] ENTRYPOINT ["/bin/bash", "-c", "echo hello"]
Step 4:
If you have followed the steps mentioned above correctly an image will be built by Kaniko and pushed to Dockerhub. Pull that image from the Dockerhub to your local machine and run to test it
$ sudo docker run -it <user-name>/<repo-name>
Unable to find image 'debuggy/helloworld:latest' locally
latest: Pulling from debuggy/helloworld
5667fdb72017: Pull complete
d83811f270d5: Pull complete
ee671aafb583: Pull complete
7fc152dfb3a6: Pull complete
Digest: sha256:2707d17754ea99ce0cf15d84a7282ae746a44ff90928c2064755ee3b35c1057b
Status: Downloaded newer image for debuggy/helloworld:latest
helloCongratulation! You successfully completed the “Hello World” section. For additional information, please see the project.
Issues with Kaniko
- Kaniko does not have v1 Registry API support
- Kaniko does not have Windows container support
- Other then official Kaniko image, running Kaniko in any other Docker image is not supported
- Kaniko might perform poor in some cases as it needs to perform snapshot of the file frequently and also due to lack of proper caching
Wrapping up
Kaniko is one of the new tool that enables creating container images in Kubernetes cluster and aims to prevent long standing reliance on the Docker daemon. It is an effective tool that enables securely creating container images in Kubernetes. It effectively accomplishes this and offers Kubernetes a reliable container image development tool. And it has done so without the usual security nightmare that came with building images on the daemon.
The official documentation for Kaniko can be found here
