Running Docker Inside and Outside of a Jenkins Container, Along With Docker-Compose — a Tiny Pipeline as Code Demonstration

Everything is Code, Versioned and Docker Docker Everywhere!

Here is a graphical walk-through demonstration of PapalUdwig’s Docker Lessons. This assumes that you already have Docker installed.

This is a tiny example of how everything can be code, stored and versioned from a SCM like GitHub. Even the pipeline definition comes from GitHub, Jenkins job definition is an empty shell pointing the GitHub repo. Any changes needed can easily accomplished in the GitHub repo itself, without having touch Jenkins.

Step 1: Launch Jenkins Docker Container

The Dockerfile I have used to push the customized Jenkins image looks ike this. All I am doing here is to install Docker and Docker Compose. Though installing Docker client is not needed as you will see later in below.

Dockerfile
FROM jenkinsci/jenkins
MAINTAINER Sreeprakash Neelakantan <sree@schogini.com>
USER root
RUN apt-get update && apt-get install -y tree nano curl sudo
RUN curl https://get.docker.com/builds/Linux/x86_64/docker-latest.tgz | tar xvz -C /tmp/ && mv /tmp/docker/docker /usr/bin/docker
RUN curl -L "https://github.com/docker/compose/releases/download/1.23.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod 755 /usr/local/bin/docker-compose
RUN usermod -a -G sudo jenkins
RUN echo "jenkins ALL=(ALL:ALL) NOPASSWD:ALL" >> /etc/sudoers
USER jenkins

Make a folder called jenkins_home on your host machine and run the below command. You will notice that the Jenkins home folder, Docker client and Docker Socket are mapped so that docker commands running within the Jenkins (job) can actually talk to the host Docker Machine.

docker run -itd -e JENKINS_USER=$(id -u) \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(pwd)/jenkins_home:/var/jenkins_home \
-v $(which docker):/usr/bin/docker \
-p 8880:8080 -p 50000:50000 \
-u root \
schogini/jenkinsci-docker-compose:v1

Let us look at the Docker container logs and wait till the Jenkins has finished its start up process.

docker logs -f d4c
*************************************************************
*************************************************************
*************************************************************

Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

d30c436a60744e8ba8d3633d07836a93

This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

*************************************************************
*************************************************************
*************************************************************

--> setting agent port for jnlp
--> setting agent port for jnlp... done
Nov 08, 2018 8:50:35 AM hudson.model.UpdateSite updateData
INFO: Obtained the latest update center data file for UpdateSource default
Nov 08, 2018 8:50:36 AM hudson.model.UpdateSite updateData
INFO: Obtained the latest update center data file for UpdateSource default
Nov 08, 2018 8:50:36 AM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Nov 08, 2018 8:50:36 AM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
Nov 08, 2018 8:50:37 AM hudson.model.DownloadService$Downloadable load
INFO: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
Nov 08, 2018 8:50:37 AM hudson.model.AsyncPeriodicWork$1 run
INFO: Finished Download metadata. 25,423 ms

There is not need to get inside the container to see the password since the host folder has been mapped to jenkins_home.

cat jenkins_home/secrets/initialAdminPassword
d30c436a60744e8ba8d3633d07836a93

Browse http://localhost:8880 and paste the password.

Click on the Install Suggested Plugins button.

Wait till all the recommended plugins have been installed.

Create the admin user.

Save and finish.

Click Start using Jenkins!

In case you logout, this is how the login screen will look like.

This is Jenkins Dashboard!

Here is the running container using docker ps

docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Ports}}'
NAMES IMAGE PORTS
thirsty_archimedes schogini/jenkinsci-docker-compose:v1 0.0.0.0:50000->50000/tcp, 0.0.0.0:8880->8080/tcp

Step 2: A Look Inside the GitHub Repository that Has Everything

Papaludwig’s repo has these files and folders, let us look at the interesting one’s for us.

git clone https://github.com/papaludwig/jenkins-docker-example.git
Cloning into 'jenkins-docker-example'...
remote: Enumerating objects: 22, done.
remote: Total 22 (delta 0), reused 0 (delta 0), pack-reused 22
Unpacking objects: 100% (22/22), done.
tree jenkins-docker-example/
jenkins-docker-example/
├── Dockerfile
├── Jenkinsfile
├── README.md
├── deploy.sh
├── docker-compose.yml
└── src
├── client
│ ├── animate.css
│ ├── app.js
│ ├── index.html
│ ├── jquery.js
│ ├── moby.png
│ ├── preloader.gif
│ └── styles.css
├── index.js
├── package.json
├── server-postgres.js
└── server.js

This is the Dockerfile that will be used to build the mobycounter image.

cat jenkins-docker-example/Dockerfile 
FROM russmckendrick/nodejs
ADD src /srv/app
WORKDIR /srv/app
RUN npm install
EXPOSE 80
ENTRYPOINT ["node", "index.js"]

Here is the docker-compose.yml file used in the build stage to deploy the microservices Docker Containers.

cat jenkins-docker-example/docker-compose.yml 
version: '2'
services:
mobycounter:
container_name: mobycounter
image: mobycounter
links:
- "redis:redis"
ports:
- "80:80"
redis:
container_name: redis
image: redis:alpine
volumes:
- "redis:/data"
volumes:
redis:
driver: local

Here is the pipeline as code, the Jenkinsfile! It has three stages.

cat jenkins-docker-example/Jenkinsfile 
node {
stage 'Checkout'
git url: 'https://github.com/russmckendrick/jenkins-docker-example.git'

stage 'build'
docker.build('mobycounter')

stage 'deploy'
sh './deploy.sh'
}

Below is the deploy script used in the deploy stage of our pipeline. In this case it deploys to a single Docker Host, this could very well be replaced by a Configuration Management script to use tools like Ansible, Chef, Puppet or SaltStack. And, the deployment target could very well be a container orchestration tool like Docker Swarm or Kubernetes. That will allow a pipeline job to deploy to hundreds of Docker Containers running on multiple Machines(Nodes) and load balanced.

cat jenkins-docker-example/deploy.sh 
#!/bin/bash
docker kill mobycounter redis > /dev/null 2>&1
docker rm mobycounter redis > /dev/null 2>&1
docker-compose up -d

Step 3 Create a Jenkins Pipeline Job

OK, let us now come back to our Jenkins dashboard and click ‘create a new job”

Proceed after entering a job name and selecting the Pipeline option.

Come to the Pipeline section and the repo url.

That is all there is! Click Build Now and watch the fun.

Pipeline is starting.

The Checkout stage

The build stage.

In case you an error, it will look like this. I had an npm install disruption first.

Reached the final deploy stage.

Successful!

Nice to see the success logs.

A full-stage view of the pipeline executions.

Let us check our docker host and see if there are running containers (from the deploy stage). Yes we can see two extra containerized services running, a Redis database and a front end Node Application.

docker-compose up -dSreeMacMin16Gdocker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Ports}}'
NAMES IMAGE PORTS
mobycounter mobycounter 0.0.0.0:80->80/tcp
redis redis:alpine 6379/tcp
thirsty_archimedes schogini/jenkinsci-docker-compose:v1 0.0.0.0:50000->50000/tcp, 0.0.0.0:8880->8080/tcp

Let us browse port 80 which the port used by the front end.

Click on the screen to paste Docker images!!

The locations of these images are saved in the Redis database so that the state of the application is persisted across deployments and restarts!

Thank you for your time, do follow for more such tiny illustrations!