How to Create a Kitchen Sink Jenkins Agent and Why It’s a Bad Idea

This is not actually my kitchen sink by the way.

Please see the updated version of this post at https://timberry.dev/posts/kitchen-sink-jenkins-agent-bad/

If you followed my last guide to spin up Jenkins in a brand new Google Kubernetes Engine cluster, you’re now probably itching to do something useful with it. The final goal will be to create a pipeline in Jenkins that manages our Terraform infrastructure, but before we get to that we’ll need to quickly pop through the looking glass into the world of custom Docker images.

Jenkins Agents

If you’re not familiar with Jenkins (oh, you lucky soul), it is an automation server that relies on agents to run jobs or complete tasks. These agents can be processes on the master server itself, or more traditionally they are separate standalone machines. Helpfully, the Helm chart we used to install Jenkins to our GKE cluster also set up the Kubernetes plugin for us. This plugin allows Jenkins to spin up agents as short-lived Kubernetes deployments inside the same cluster, so we don’t actually have to configure any agent machines or worry about managing them.

By default Jenkins images will use the jenkins/jnlp-slave docker image. Will this image contain the tools necessary for us to complete the tasks in our pipeline? Probably not. However with Docker it’s easy to inherit an image and build your own custom version of it.

The Custom Image

Create a new directory on your local machine and grab the jenkins-slave file that the original image uses, then create the following Dockerfile:

As you can see, we build on an existing image from Jenkins, then add the tools we want: Docker, Kubectl, Terraform and the GCloud SDK. Note that there are some hard-coded URLs and version numbers in there, which is not really best practice.

Let’s also create the docker-entrypoint.sh required for Docker-in-Docker to work:

What’s Docker-in-Docker I hear you say? Well, recall that our Jenkins agent is running within Docker itself (albeit orchestrated by Kubernetes). If we want to use this agent to create Docker images, we end up running Docker inside Docker itself. There are new and better ways of doing this and one day I’ll find the time to learn them!

Okay, with those files now in place, let’s build our custom image:

docker build -t my-custom-agent .

You should now have a custom Jenkins agent docker image ready to go. We can push it to Docker Hub, or Google Container Registry, and update our Jenkins Helm deployment to use this image for agent deployments. But wait:

Is this how I parking?

We’re Doing It Wrong

There is a better way. Creating a kitchen-sink Jenkins agent may be fine for experimenting with pipelines when you need every tool at your disposal quickly, but it’s actually an anti-pattern. Why?

• Containers should be as lightweight as possible
• Segregating tools allows you to manage them separately
• Pipelines can use multiple containers inside the same pod

The last point there is worth restating: You can define multiple containers within the same pipeline to carry out the various tasks and stages.

Thanks to the way the Kubernetes plugin works, each container will share the Jenkins home directory (in other words, the workspace of the pipeline) as they all exist within the same pod in your Kubernetes cluster.

In the final part of this series we’ll create our Terraform pipeline, and use off the shelf lightweight containers to run each stage. In the meantime, I hope my experience of learning to build a custom agent and then discovering why I shouldn’t use it was helpful to you.