CI and CD with Jenkins — Part 1

Published in

walkin

9 min readOct 9, 2019

In this article, we will discuss how to set up CI and CD pipeline using Github, Jenkins, and Docker. This will be divided into two parts:

Part-1: CI Pipeline setup: In this part, we will cover the installation and setup of Jenkins and integrating Jenkins with Github.

Part-2: CD Pipeline setup: In this part, we will cover setting up Github repo and build a simple Docker image and deploy it on a server using Jenkins.

DevOps is a set of practices that automates the processes between software development and IT teams, in order that they can build, test, and release software faster and more reliably. The concept of DevOps is founded on building a culture of collaboration between teams that historically functioned in relative siloes. The promised benefits include increased trust, faster software releases, the ability to solve critical issues quickly, and better manage unplanned work.

A CI tool helps us to detect problems or bugs, as early as possible, in the development life cycle. Since the entire code base is integrated, built and tested constantly, the potential bugs and errors are caught earlier in the life cycle resulting in better quality software and reducing the overall cost.

The main use of CI looks like this:

Developer commits code to the central repository which can be hosted on GitHub or GitLab.
Jenkins can be connected to this repository with a hook or Github plugin or Gitlab plugin. Whenever a commit is detected, Jenkins pulls the changes and builds the codebase.
Unit tests are run along with code quality and code coverage metrics to evaluate the code quality.
Automated deployment to the production server after testing is done. If you get any error, it will notify the admin.

In this article, we will discuss how to set up CI and CD pipeline using Github, Jenkins, and Docker

Installation and Setup of Jenkins

Jenkins is a popular open-source tool to perform continuous integration and build automation. The basic functionality of Jenkins is to execute a predefined list of steps, e.g. to compile Java source code and build a JAR from the resulting classes. The trigger for this execution can be time or event-based. For example, every 20 minutes or after a new commit in a Git repository.

Possible steps executed by Jenkins are for example:

perform a software build using a build system like Apache Maven or Gradle
execute a shell script
archive a build result
running software tests

Jenkins monitors the execution of the steps and allows them to stop the process if one of the steps fails. Jenkins can also send out notification in case of a build success or failure.

Jenkins can be extended by additional plugins. For example, you can install plugins to support building and testing Android applications.

Prerequisites

We are going to use Ubuntu 18.04 server for this Jenkins. It should be configured with a non-root sudo user and a firewall by following the Ubuntu 18.04 initial server setup guide. It is recommended to take a server that has at least 2GB of RAM. Jenkins server should be able to access(SSH) the server where we need to deploy the application.

Before installing Jenkins, check whether Maven 3.*.* and Java is installed or not in the server. If Java is not installed, go through the below article to install java:

https://www.linuxbabe.com/ubuntu/install-oracle-java-8-openjdk-11-ubuntu-18-04-18-10

Step 1 — Installing Jenkins

First, we’ll add the repository key to the system.

$ wget -q -O — https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -

When the key is added, the system will return OK. Next, we’ll append the Debian package repository address to the server’s sources.list:

$ sudo sh -c ‘echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’

When both of these are in place, we’ll run update so that apt-get will use the new repository:

$ sudo apt-get update

Finally, we’ll install Jenkins and its dependencies, including Java:

$ sudo apt-get install jenkins

If any dependency is missing run below command and after that try again installation command

$ sudo apt-get -f install

Upgrade:

You can update to the later version of Jenkins (when it comes out) by running the following commands:

$ sudo apt-get update$ sudo apt-get install jenkins

Now that Jenkins and its dependencies are in place, we’ll start the Jenkins server.

Step 2 — Starting Jenkins

We will start the Jenkins with below command

$ sudo service jenkins start

Since above command doesn’t display output, we’ll use its status command to verify that it started successfully:

$ sudo service jenkins status

If everything went well, the beginning of the output should show that the service is active and configured to start at boot:

Output:

jenkins.service — LSB: Start Jenkins at boot timeLoaded: loaded (/etc/init.d/jenkins; bad; vendor preset: enabled)Active: active (exited) since Fri 2018–06–29 06:43:06 UTC; 1min 4s agoDocs: man:systemd-sysv-generator(8)```

Now that Jenkins is running, we’ll adjust our firewall rules so that we can reach Jenkins from a web browser to complete the initial set up.

Step 3 — Opening the Firewall

By default, Jenkins runs on port 8080, so we’ll open that port using ufw:

$ sudo ufw allow 8080

We can see the new rules by checking UFW’s status.

$ sudo ufw status

We should see that traffic is allowed to port 8080 from anywhere:

Output:

Status: activeTo Action From
 — — — — — — 
OpenSSH ALLOW Anywhere
8080 ALLOW Anywhere
OpenSSH (v6) ALLOW Anywhere (v6)
8080 (v6) ALLOW Anywhere (v6)

Note: If the firewall is inactive, the following commands will enable it and then make sure that OpenSSH is allowed.

$ sudo ufw enable$ sudo ufw allow OpenSSH

Now that Jenkins is installed and the firewall allows us to access it, we can complete the initial setup.

Step 4 — Setting up Jenkins

To set up our installation, we’ll visit Jenkins by opening the web browser and open the link below. Make sure you change ip_address_or_domain_name with your the server domain name or IP address :

http://ip_address_or_domain_name:8080

We will see “Unlock Jenkins” screen, which displays the location of the initial password

In the terminal window, we can use the cat command to display the password

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

We’ll copy the password from the terminal and paste it into the “Administrator password” field, then click “Continue”. The next screen presents the option of installing suggested plugins or selecting specific plugins.

We’ll click the “Install suggested plugins” option, which will immediately begin the installation process.

When the installation is complete, we’ll be asked to set up the first admin user. It’s possible to skip this step and continue as admin using the initial password we used above, but we’ll create the user.

Once the first admin user is in place, you should see a “Jenkins is ready!” confirmation screen.

Click “Start using Jenkins” to visit the main Jenkins dashboard. Jenkins has been successfully installed.

Github access to Jenkins

To access Github repositories we need to configure Github in Jenkins. Select Manage Jenkins Option present on the left side.

Select Configure System which under Manage Jenkins

Go to the Github section and select Add Github Server.

Next enter the Name and API URL.

Then select Add credentials, it will show a Jenkins Credentials provider popup. In that select Secret Text option for Kind and Global option for Scope. For the Secret option, we need to get Auth token from Github.

Login to Github account and select the account setting. After that select Developer setting which presents on the left side.

Then select Personal access token and click Generate token present on the right side. Enter token description and select scopes. Jenkins’ scope requirements depend on the task/s you wish to perform:

admin:repo_hook — For managing hooks at GitHub Repositories level including for Multibranch Pipeline
admin:org_hook — For managing hooks at GitHub Organizations level for GitHub Organization Folders
repo — to see private repos. Please note that this is a parent scope, allowing full control of private repositories that include:
repo:status — to manipulate commit statuses
repo:repo_deployment — to manipulate deployment statuses
repo:public_repo — access to public repositories
read:org and user:email — recommended minimum for GitHub OAuth Plugin scopes.

After selecting scopes, then click Generate token. It will generate an alphanumeric token. Copy the token and paste it in Secret option in Jenkins credentials provider and enter ID as Github and Click Add.

After adding the credentials, select the credential with id Github and click Test connection. It will show Credentials: verified for the user. Then apply and save the settings

CI & CD

Plugin Setup In Jenkins and Github

We have given Github access to Jenkins next we need to Install Github Plugin in Jenkins and Jenkins plugin in Github for Continuous Integration. Select Manage Jenkins on the left side of the home page.

Select Manage Plugins.

Select Available option and search below plugins and install them and then restart the Jenkins

GitHub
GIT server
GitHub Authentication
GitHub Integration

Login to the Github account. Open project repository and go to setting. In setting select Integration & service. Click on Add Service and add ‘Jenkins (GitHub plugin)’.

Then enter Jenkins hook URL. In place of “jenkins_url” enter your Jenkins IP or DNS name. Next, select the checkbox Active and click Add service.

After adding click Jenkins (GitHub plugin). Then click Test service. It will send test payload.

If you go back, there will be a green tick mark before Jenkins.

Project Setup In Jenkins

Select New Item which is on the left side

Enter your Project Name and select Freestyle Project and click OK.

In the general section, select the Discard old builds checkbox. It will discard old build from Jenkins. Enter Days to keep builds and Max # of builds to keep. Next, select the GitHub project checkbox and enter your Github project URL in the input box.

Next in the Source code management section, select Git and enter the Github project URL and enter the branch name after entering “*/”.

Next in the Build Trigger section, We can select either Build periodically or GitHub hook trigger for GITScm polling(Previously named as “Build when a change is pushed to GitHub”) or both. Here we will select only GitHub hook trigger for GITScm polling. This will trigger build whenever someone pushes changes to Github.

Every time you publish your changes to Github, GitHub will trigger your new Jenkins job. Once the GitHub plugin is fully installed and integrated into your Jenkins project, you have completed a very crucial step towards the full CI process.

Conclusion of part-1:

In this part-1, we have discussed the CI pipeline setup and in part-2 we will discuss the CD pipeline setup. Below I have listed a few famous tools that can be used for CI:

- GitlabCI- TeamCity- CircleCI- Bamboo- Buddy- TravisCI- Codeship- Wercker- Nevercode- BuildKite