Creating a Jenkins container with persistent data using Docker Compose

Hello there! I’m so glad you decided to read this article because today we’ll be discussing a tool that will change your life (assuming you’re creating containers all day lol). Today I will be giving a brief overview and tutorial of Docker Compose, an extremely popular tool used by DevOps and Cloud Engineers everywhere to simplify and automate the deployment of their containers. For our example we’re going to build a Jenkins container. As always, before we begin, I’ll be defining some of the terms used in this article.

Docker Compose -

Docker compose is a tool used to define and execute the deployment of a multi-container environment. It takes its instructions from the docker compose file. Written in YAML, the docker compose file lays out steps to build and run the containers and any attached networks or volumes. The real kicker is that once you have your docker compose file written, you can bring up or tear down entire environments using only a single command. Capable of handling many containers at once, it is the perfect tool to create, manage and tear down your microservice applications.

Docker Volume -

Simply put, a volume in docker is where you would store any data you wish to re-use at a later time. By their nature, containers are what’s known as ephemeral. Meaning that they are stateless and any data saved to a running instance is lost once that container is stopped. The way around this is to attach a volume to your container. This will allow you to store your data outside of the container itself, allowing it to be re-used again and again by any subsequent containers. Your data “persists” through the container’s shutdown.

Jenkins -

Jenkins is a Continuous Integration (CI) tool used by developers to build and test their applications. However, we will not be using it in such a way. We’re only going to use it to confirm the data stored in our volume persisted.

That’s all we have this week for new terms. Now, we’ll move on to defining our objectives.

Objectives -

Foundational:

Using the CLI

1. Pull the Jenkins LTS image from Docker Hub.
2. Create a Docker volume for Jenkins to persist its data.
3. Run the container with the volume and correct port mapped to the container.
4. Retrieve the Admin password.
5. Login and create a new user.
6. Launch a new Jenkins container using the same volume and verify that the data has persisted by logging in as the newly created user.

PREREQUISITES:

DOCKER HUB ACCOUNT — Docker hub is the registry we will be pulling our official image from. You will need to have an account. If you do not already have one, you can create one for free by navigating to https://hub.docker.com/signup.

LINUX SERVER WITH DOCKER INSTALLED— I will be using an EC2 Instance with Ubuntu 22.04 installed for this tutorial. You will need a similar setup to follow along.

IDE OF YOUR CHOICE (Optional) -

I will be using Visual Studio Code in this tutorial but feel free to use whatever IDE you like best. Optionally, you could use a built-in text editor instead.

Ok, now it’s time to get our hands dirty!

Step 1:

Head to DockerHub and search for Jenkins

Notice how this image is deprecated. Better not use that one. Head to the suggested image called jenkins/jenkins.

Copy the pull command on the right side of the page.

Paste it in your terminal. The output should look like this:

You can verify image is pulled with docker image ls.

docker image ls

Step one complete!

Step 2:

Now we need to add a volume. Head to Docker docs and search for “Volumes’’. Scroll down to the create volume example and copy it.

Paste it into the terminal and replace “my-vol” with the name you wish to call your volume.

docker volume create <volume-name>

Run the docker volume ls command to verify the volume has been created.

docker volume ls

Step 2 down! Let’s keep it moving.

Step 3:

We need to figure out the correct port number to use. To do that, head back over to the jenkins/jenkins DockerHub page and click on the link for the documentation.

This will take you to the README file for the image. Right near the top of the file, copy the docker run command that specifies a volume. This example already provides us with the mountpoint of the volume inside the container, as well as the port numbers we need to use. We simply need to replace the info where needed and omit the unnecessary options. Change the name of the volume to whatever you called yours. Remove the tags from the image name at the end of the command. You can also remove -p 50000. This port is only needed when you are connecting agents, which we do not have to worry about for this project. The “-v” option is how we are telling docker to mount the volume to the container.

The cleaned up run command should look like this:

docker run -d -v week-17-vol:/var/jenkins_home -p 8080:8080 jenkins/jenkins

Run the command and then run docker ps to verify the container is running.

Step 3 done! Keep knocking them out!

Step 4:

Next we need to retrieve the admin password. We can easily do that by running the docker logs command and specifying the new container. You only need to type the first few characters of the Container ID into the command.

docker logs <container-id>

Near the bottom of the output you will see the initial admin password displayed in plain-text.

Copy that password and save it somewhere safe. We’ll need it in just a moment when we sign in.

Step 4 complete!

Step 5:

To login we’ll use our browser. More than likely in the search bar you can enter “localhost:8080”, but since I am working on an EC2 Instance, I’ll copy my public ip address and enter that into the search bar followed by “:8080”.

Paste the initial admin password that you copied into the “Administrator password” field.

Now you’ll need to go through the initial setup.

Once at the Dashboard, click on “Manage Jenkins”. Then click on “Users”.

Click on “Create User”.

Create a new user. Be sure to save the login credentials because we will be using those in just a second.

Step 5 is finished! You’re nearly there!

Step 6:

First, let’s stop our running container. To do that we’ll use the docker container stop command.

docker container stop <container-id>

Run docker ps to verify the container is no longer running.

To test our volume and ensure that our data has persisted, we’ll create a new container using the same volume and try logging in as the newly created user. To do that, press the up arrow on the keyboard a few times until you are at the docker run command. All we are going to do is add a name to the command to differentiate between the 2 containers.

In the browser, refresh the tab for “localhost:8080” (or in my case, “public-ip:8080”). This will take you to the Jenkins login page. Login with the new user credentials.

As you can see, we are logged into the new user account, confirming our data has persisted due to the volume we created. Easy peezy.

That’s it for the foundational portion. Let’s move onto something a bit more advanced.

ADVANCED:

OBJECTIVES:

1. Instead use Docker Compose to do the steps in the Foundational.

So now that we understand how to pull images and turn them into containers while simultaneously creating and attaching a volume, let’s take it one step further and do all that again, only this time we’ll utilize Docker Compose.

First we’ll run a docker system prune command to remove all unused containers, images, networks and we can also specify volumes.

docker system prune --volumes 

Now let’s create a docker compose file and fill it with all the info we’ll need to get started.

Composing. Get it? lol.

As you can see, the docker compose file specifies the same image, port numbers and volume as the docker run command we ran earlier. The main difference to point out is in the “services” block. Do you see where it says “new-jenkins-container”? That is the name we will be giving to the container when we run it. Inside the docker compose file, when you see “services”, think “container name”.

Save the file.

Now we’ll launch the build and run process and expose the correct port numbers and attach the correct volume. We’ll do this all with a single command, “docker compose up”

docker compose up

I restarted my EC2 instance while writing this article, so I need to go back and grab my public ip address again.

Now we’ll go through the Jenkins setup one more time and create a new_user.

Back in the terminal, stop the container with ctrl+c and then type “docker compose down”.

docker compose down

Now we’ll create a 2nd docker compose file. Copy the contents of the original docker compose file and paste them into the 2nd docker compose file. Mine is called docker-compose-new_user.yml.

Update the services block to change the name of the container to “new_user-jenkins-container”.

Save the file. This time when we run the docker compose up command, we’ll utilize the “-f” flag to specify which file to use as the docker compose file. We’ll point the flag at docker-compose-new_user.yml.

docker compose -f docker-compose-new_user.yml up

And just as before, we’re taken to the Jenkins login screen. Try logging in with the “new_user” credentials you made.

Success! The fact that you can log in as new_user shows us our data persisted in the volume, even when attached to a different container.

And that’s all there is to it! If you made it this far give yourself a big pat on the back. I hope this tutorial demonstrates the increased efficiency and control made possible by docker compose. Before we end though we need to clean up. Time to tear it all down again! After stopping the container with ctrl+c run the docker compose down command on last time.

I hope you had as much fun reading this article as I had making it. Until next time… see ya!

GitHub Gists:

Below are the secret gists to my 2 docker compose files.