Setup Jenkins for your application from the private github repo with docker

Although I have experience with Docker and orchestrated tool such as Kubernetes, I still struggle with setup and running CICD with Jenkins. In this post, I just summarised what it’s been working for me.

Tech stack: CICD with Jenkins, Ubuntu 20.04, Docker and github repo as the object of this process.

Step #1: Build and compile Jenkins

The jenkins image is built with the tool itself, without plugins. Plugins can be setup when you first running the service. However, it’s good to customize the plugins and install the tools at compiling time.

Example of my dockerfile for Jenkins with plugins as such (plugins.txt)

FROM jenkins/jenkins:latest

LABEL MAINTAINER="Your name <your-email>"

# Disable the Setup Wizard
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false

# Config file for Jenkins
ENV CASC_JENKINS_CONFIG /var/jenkins_home/casc.yaml

# All the plugins required for Jenkins pipeline
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt

# Settings for Jenkins
COPY casc.yaml /var/jenkins_home/casc.yaml

# Install all the plugins
RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt

# Update and install the docker so can build image and everything else
RUN apt-get update && curl -sSL https://get.docker.com/ | sh

Sample plugins for jenkins

ant:latest
antisamy-markup-formatter:latest
authorize-project:latest
build-timeout:latest
cloudbees-folder:latest
configuration-as-code:latest
credentials-binding:latest
email-ext:latest
git:latest
github-branch-source:latest
gradle:latest
ldap:latest
mailer:latest
matrix-auth:latest
pam-auth:latest
pipeline-github-lib:latest
pipeline-stage-view:latest
ssh-slaves:latest
timestamper:latest
workflow-aggregator:latest
ws-cleanup:latest
docker-workflow:latest
docker-compose-build-step:latest

Sample settings for Jenkins in casc.yaml file

jenkins:
  securityRealm:
    local:
      allowsSignup: false
      users:
       - id: ${JENKINS_ADMIN_ID}
         password: ${JENKINS_ADMIN_PASSWORD}
  authorizationStrategy:
    globalMatrix:
      permissions:
        - "Overall/Administer:admin"
        - "Overall/Read:authenticated"
  remotingSecurity:
      enabled: true
security:
  queueItemAuthenticator:
    authenticators:
    - global:
        strategy: triggeringUsersAuthorizationStrategy
unclassified:
  location:
    url: http://172.21.149.187:8080/

With the Dockerfile above, you can compile in your workstation with

$ docker build -t jenkins:customise .

Step #2: Start up the Jenkins

I prefer to start the docker service with docker-compose file, therefore, the sample docker-compose file as such

version: '3'
services:
    jenkins-docker:
        image: jenkins:customise
        restart: unless-stopped
        user: root
        privileged: true
        environment:
            - JENKINS_ADMIN_ID=admin
            - JENKINS_ADMIN_PASSWORD=password
        ports:
            - "8080:8080"
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock

Then starting service is simple by using this command

docker-compose up

You can access the address http://localhost:8080/ with username and password specified in the docker environment variables.

Step #3: Config the jenkins to allow

• Adding credentials to access the private github repo

At your workstation/computer, create the SSH key, follow the link here:

or using this command (replace parameter to -C with your email)

ssh-keygen -t ed25519 -C "your-email"

Go to the dashboard -> Manage Jenkins -> Credentials -> Global -> Global credentials (unrestricted)

Next

Next

Add credentials -> Select “SSH username with private key”

ID=<example_github>

Description=<Optional, can describe the id>

Private key=<Enter directly from your private SSH key>

• Adding the private git repo to the known-hosts

Open the other terminal to ssh into the jenkins containers

1. Generate the ssh key pair inside the container
2. Adding the public key to the github repo -> Deploy keys

$ ssh-keygen -t ed25519 -C "your-email"
<NO paraphrase>

$ docker exec -it <jenkins-container-name> bash
root@4ece49792c24:/# git ls-remote -h git@github.com:vthily/<your-git-repo>.git HEAD
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
root@4ece49792c24:/#

Adding deploy keys to your git repo:

• Adding credentials to access and push docker image to the docker hub: You can do the same with GitHub id, but with username and password.

Step #4: Create the first CI pipeline to auto build

Add the description, and input your pipeline script to the box as below:

Sample pipeline script

#!groovy

pipeline {
    environment {
        workingDir = "my-first-pipeline"
        registry = "docker-user/docker-repo"
        dockerImage = ''
        
    }
  
    agent any
    stages {
        stage("Clean up") {
            steps {
                deleteDir()
            }
        }
        stage("Clone Repo") {
            steps {
                git([url: 'git@github.com:<your-git-user>/<your-private-git-repo>.git', branch: '<your-selected-branch', credentialsId: 'vthily_github'])
            }
        }
        stage("Build") {
            steps{
                script {
                    dockerImage = docker.build registry + ":$BUILD_NUMBER"
                }
            }
        }
        stage("Test") {
            steps {
                dir(workingDir) {
                    // List all the docker images on the agent workstation
                    sh "docker image ls"
                    
                    // Remove the built image
                    sh "docker rmi $registry:$BUILD_NUMBER"
                    
                    // Delete the build cache objects
                    sh "docker system prune -f"
                }
            }
        }
    }
}

If the build fails, you can see it from the block in Stage View

Navigate to Console Output, example with build fails:

the successful build

the status of all builds for the pipeline is showing in “W(eather)” icon, rain if too many failure builds, cloud if equally distributed, or sunny if more success builds than failure ones.

Wish you luck with Jenkins setup and building.