Ubuntu VM/Azure: Install & Run Jenkins in Docker/Docker Compose with persistent container volume located on the external Cloud Data Disk
As you can see from the title that’s a lot of configuration going on here. Basically, I am trying to set up a very simple development workflow using very fully customizable (and cheap) configuration.
Why in Windows Azure? I have a running commercial project and paid a subscription so why not also use it for development purposes.
But before diving into the technical details let’s try to visualize it. That’s what I wanted to achieve:
Ubuntu 18.04 and disk storage
Actually, the configuration is quite straight forward. We have Ubuntu 18.04 virtual machine running in Microsoft Azure. 1TB external storage mounted as /apps/data (about the steps to mount it I will talk a bit later). This storage is needed to keep our configuration and data persistent. Hence, I configured it as managed and premium storage. In terms of Microsoft Azure, shortly, managed disk storage means it is highly durable and available, has integration with availability zones (to avoid a single point of failure) and eligible for Azure Backup support. It’s also Premium SSD which deliver high performance and low-latency disk support for VMs.
VM with Docker/Docker Compose and mounted data storage
As a next step, we need to setup disk for our VM. Check this article on how to add a data disk to Linux VM. For the instructions to actually mount the disk to Linux VM I used these 3 articles:
How to mount Azure Blob storage as a file system on Linux
Blobfuse is a virtual file system driver for Azure Blob storage. Blobfuse allows you to access your existing block blob…
Add a data disk to Linux VM using the Azure CLI
This article shows you how to attach a persistent disk to your VM so that you can preserve your data — even if your VM…
Creating & Mounting new drives in Ubuntu / Azure
Leading note: The drive creation is based on using Azure with an Ubuntu 14.04 LTS release. This stuff doesn’t change…
My final fstab configuration for the disk looked like this:
UUID=uuid-of-disk /apps/data ext4 defaults,nofail 0 1
Running Jenkins in Docker/Docker Compose with external volume
Now, when we have all configured VM with Docker/Docker Compose, it is time to add our Docker configuration. Our Docker Compose file looks like this:
The most important and interesting part here is how we setup volumes. But about this a bit later. Let’s see Dockerfile that we use to build an actual image:
FROM jenkinsci/blueoceanUSER rootRUN addgroup jenkins && adduser -D -G jenkins jenkinsUSER master
It’s based on jenkinsci/blueocean and what we do here is a little trick to deal with user permissions.
As you know we will use persistent volume to keep our Jenkins data and it will be stored in mounted storage. The thing is actual files in the volume is stored in a host machine and used in Docker containers. To avoid permission errors you need to have proper user and group configuration. And that’s why in Dockerfile we add a new user called jenkins with jenkins group. Official image from is also using this user. More about this you can read here.
And in the host machine, for the directory /apps/data/jenkins-data, we have jenkins:jenkins user and group. To do this you can use the following command:
chown -R jenkins:jenkins /apps/data/jenkins-data
Hence, when you mount this directory to the container (see docker compose conf) you have the same user and group and as a result, you do not get permission denied error.
In Ubuntu 18.04 to create this user and group you can use following commands:
useradd -G jenkins jenkins
And finally to our docker container use the following command
docker-compose up -d
restart: always configuration in docker compose will make sure that it is always running after system or docker reboot.
I tried very hard to keep this article short and to the point. I highly recommend checking articles on Docker volumes and how they work. Please check following links for this:
Also, once you installed Docker in Linux machine make sure to check post-installation steps here: https://docs.docker.com/install/linux/linux-postinstall/.
If you’ve any feedback, feel free to tell me what I can improve, or provide better examples.
Feel free to clap as many times as you like :)