Setup Jenkins for your application from the private github repo with docker
Although I have experience with Docker and orchestrated tool such as Kubernetes, I still struggle with setup and running CICD with Jenkins. In this post, I just summarised what it’s been working for me.
Tech stack: CICD with Jenkins, Ubuntu 20.04, Docker and github repo as the object of this process.
Step #1: Build and compile Jenkins
The jenkins image is built with the tool itself, without plugins. Plugins can be setup when you first running the service. However, it’s good to customize the plugins and install the tools at compiling time.
Example of my dockerfile for Jenkins with plugins as such (plugins.txt)
FROM jenkins/jenkins:latest
LABEL MAINTAINER="Your name <your-email>"
# Disable the Setup Wizard
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false
# Config file for Jenkins
ENV CASC_JENKINS_CONFIG /var/jenkins_home/casc.yaml
# All the plugins required for Jenkins pipeline
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
# Settings for Jenkins
COPY casc.yaml /var/jenkins_home/casc.yaml
# Install all the plugins
RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt
# Update and install the docker so can build image and everything else
RUN apt-get update && curl -sSL https://get.docker.com/ | sh
Sample plugins for jenkins
ant:latest
antisamy-markup-formatter:latest
authorize-project:latest
build-timeout:latest
cloudbees-folder:latest
configuration-as-code:latest
credentials-binding:latest
email-ext:latest
git:latest
github-branch-source:latest
gradle:latest
ldap:latest
mailer:latest
matrix-auth:latest
pam-auth:latest
pipeline-github-lib:latest
pipeline-stage-view:latest
ssh-slaves:latest
timestamper:latest
workflow-aggregator:latest
ws-cleanup:latest
docker-workflow:latest
docker-compose-build-step:latest
Sample settings for Jenkins in casc.yaml file
jenkins:
securityRealm:
local:
allowsSignup: false
users:
- id: ${JENKINS_ADMIN_ID}
password: ${JENKINS_ADMIN_PASSWORD}
authorizationStrategy:
globalMatrix:
permissions:
- "Overall/Administer:admin"
- "Overall/Read:authenticated"
remotingSecurity:
enabled: true
security:
queueItemAuthenticator:
authenticators:
- global:
strategy: triggeringUsersAuthorizationStrategy
unclassified:
location:
url: http://172.21.149.187:8080/
With the Dockerfile above, you can compile in your workstation with
$ docker build -t jenkins:customise .
Step #2: Start up the Jenkins
I prefer to start the docker service with docker-compose file, therefore, the sample docker-compose file as such
version: '3'
services:
jenkins-docker:
image: jenkins:customise
restart: unless-stopped
user: root
privileged: true
environment:
- JENKINS_ADMIN_ID=admin
- JENKINS_ADMIN_PASSWORD=password
ports:
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
Then starting service is simple by using this command
docker-compose up
You can access the address http://localhost:8080/ with username and password specified in the docker environment variables.
Step #3: Config the jenkins to allow
- Adding credentials to access the private github repo
At your workstation/computer, create the SSH key, follow the link here:
or using this command (replace parameter to -C with your email)
ssh-keygen -t ed25519 -C "your-email"
Go to the dashboard -> Manage Jenkins -> Credentials -> Global -> Global credentials (unrestricted)
Next
Next
Add credentials -> Select “SSH username with private key”
ID=<example_github>
Description=<Optional, can describe the id>
Private key=<Enter directly from your private SSH key>
- Adding the private git repo to the known-hosts
Open the other terminal to ssh into the jenkins containers
- Generate the ssh key pair inside the container
- Adding the public key to the github repo -> Deploy keys
$ ssh-keygen -t ed25519 -C "your-email"
<NO paraphrase>
$ docker exec -it <jenkins-container-name> bash
root@4ece49792c24:/# git ls-remote -h git@github.com:vthily/<your-git-repo>.git HEAD
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
root@4ece49792c24:/#
Adding deploy keys to your git repo:
- Adding credentials to access and push docker image to the docker hub: You can do the same with GitHub id, but with username and password.
Step #4: Create the first CI pipeline to auto build
Add the description, and input your pipeline script to the box as below:
Sample pipeline script
#!groovy
pipeline {
environment {
workingDir = "my-first-pipeline"
registry = "docker-user/docker-repo"
dockerImage = ''
}
agent any
stages {
stage("Clean up") {
steps {
deleteDir()
}
}
stage("Clone Repo") {
steps {
git([url: 'git@github.com:<your-git-user>/<your-private-git-repo>.git', branch: '<your-selected-branch', credentialsId: 'vthily_github'])
}
}
stage("Build") {
steps{
script {
dockerImage = docker.build registry + ":$BUILD_NUMBER"
}
}
}
stage("Test") {
steps {
dir(workingDir) {
// List all the docker images on the agent workstation
sh "docker image ls"
// Remove the built image
sh "docker rmi $registry:$BUILD_NUMBER"
// Delete the build cache objects
sh "docker system prune -f"
}
}
}
}
}
If the build fails, you can see it from the block in Stage View
Navigate to Console Output, example with build fails:
the successful build
the status of all builds for the pipeline is showing in “W(eather)” icon, rain if too many failure builds, cloud if equally distributed, or sunny if more success builds than failure ones.
Wish you luck with Jenkins setup and building.