Using Docker Containers as Jenkins Build Slaves

Photo by CJ Dayrit on Unsplash

WHY?

Reasons:

1. Using ephemeral Docker containers as Jenkins build slaves for various stages result in better resource utilization.
2. Spinning up a new container takes less than a minute, every build spins up a new container, builds the project, and is destroyed. This way, you can reduce the number of static Jenkins build VMs.
3. You can turn version of different dependencies into static (Docker Image), eliminating the risk of breakage in CI/CDs due to any version change on the Jenkins server.

4. Cleaner way of working with CI/CDs.

So, recently I came across the 3rd point mentioned above that picking all the dependencies from our Jenkins server is not promising, in case the version of any dependency is changed from the server by someone and BOOM! this will cost in the failure of all integration pipelines, Hence we are at risk.

So, why not package our dependency into a docker image, configure Jenkins to spin a container on the Docker host machine, and execute the job inside that container?

Great right 😃

Let us implement this!

I will walk you through the steps for configuring Docker containers as build slaves.

Step 1. Configure Docker host with Remote API

The first step we should do is set up a Docker host. Jenkins server will connect to this host for spinning up the slave containers. Jenkins master connects to the docker host using REST APIs. So we need to enable the remote API for our docker host. Make sure that the connection is open for the following ports to accept connections from Jenkins master.

Port 4243: Docker Remote API port

HostPort Range: 32768 to 60999, used by Docker to assign a host port for Jenkins to connect to the container

• Login to server 2 where Docker is installed and open the docker service file /lib/systemd/system/docker.service. Search for ExecStart and replace that line with the following:

ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock

• Reload and restart Docker service

sudo systemctl daemon-reload
sudo service docker restart

• Go to Jenkins Host server and execute

curl http://10.10.25.51:4243/version

Replace your Host IP with 10.10.25.51. You should receive output in JSON format as shown below.

That’s it! Now we will move towards creating a Jenkins agent docker image.

Step 2. Write Dockerfile with Dependencies

Minimum configuration required by Docker image to act as a slave:

1. sshd service running on port 22.
2. Jenkins user with password.
3. All the required application dependencies for the build. For example, for a java Gradle project, you need to have git, java, and Gradle installed on the image.

Step 3(i). Configure Jenkins Server with Docker Plugin

1. Head over to Jenkins Dashboard –> Manage Jenkins –> Manage Plugins.

2. Under the Available tab, search for “Docker” and install the docker cloud plugin and restart Jenkins. Here is the official plugin site.

3. Once installed, head over to Jenkins Dashboard –> Manage Jenkins –>Configure system.

4. Under “Configure System”, if you scroll down, there will be a section named “cloud” at the last. There you can fill out the docker host parameters for spinning up the slaves.

Step 3(ii): Under docker, you need to fill out the details as shown in the image below.

Note: Replace “Docker URI” with your docker host IP. For example, tcp://10.10.25.51:4243 You can use the “Test connection” to test if Jenkins is able to connect to the Docker host.

6. Now, from the “Docker Agent Template” dropdown, click the “Add Docker template” and fill in the details as shown in the image given below and save the configuration.

Note: Replace “Docker Image” with your docker image. Meaning, dockerslave:6.0.0 should be replaced by your docker image address.

Step 4. Create a Jenkins job to test

In order to test this implementation, you can either create a freestyle job or can write a pipeline. I am using a freestyle job for easy understanding.

Configure a freestyle job as shown in the picture below.

Click on build now, you will notice that the build goes into the pending state for some seconds, it is because of the time taken by the host server to spin up the docker container.

But soon it will start executing like shown in the below image:

Click on console output to check the address of the docker slave agent.

Step 5. Check on the Docker server for the new container

Log in to Docker host server and check for running containers

docker ps

Here, you will see a new container spined.

Step 6. Once the job is successful — Verify that the container no longer exists on the server.

Again, check for running containers by docker ps command, now you will notice that the container no longer exists.

We are done, you just successfully configured a docker container as Jenkins build slave 😃

Thanks for Reading!

I hope that you found this article to be helpful. I have configured to use docker containers as slaves to run the build stage of the Jenkins pipeline. Let me know your use case in the comments down below. Happy Reading!