CI/CD Pipeline Using Git, Jenkins & Kubernetes
In this article, I will demonstrate Continuous Integration & Continuous Deployment using Jenkins, Git, and Kubernetes. Also before begin let's see a little bit of overview of these three popular technologies.
Jenkins :
Jenkins is an open-source automation tool written in Java with plugins built for Continuous Integration purposes. Jenkins is used to build and test your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build.
- To know more about Jenkins: https://www.jenkins.io/
Git :
Git is a distributed version control system for tracking changes in source code during software development. It is designed for coordinating work among programmers, but it can be used to track changes in any set of files. Its goals include speed, data integrity, and support for distributed, non-linear workflows.
- To know more about Git: https://git-scm.com/about
Kubernetes :
Kubernetes is an open-source container-orchestration system for automating computer application deployment, scaling, and management. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. It aims to provide a “platform for automating deployment, scaling, and operations of application containers across clusters of hosts”. It works with a range of container tools, including Docker.
- To know more about Kubernetes: https://kubernetes.io/docs/concepts/overview/what-is-kubernetes
So, Let's see the Problem Statement :
1. Create a container image that’s has Jenkins installed using Dockerfile Or You can use the Jenkins Server on RHEL 8/7.
2. When we launch this image, it should automatically start the Jenkins service in the container.
3. Create a job chain of job1, job2, job3, and job4 using build pipeline plugin in Jenkins.
4. Job1: Pull the GitHub repo automatically when some developers push the repo to GitHub.
5. Job2 :
I) By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )
II) Expose your pod so that the testing team could perform the testing on the pod.
III) Make the data to remain persistent (If the server collects some data like logs, other user information ).
IV) Job 3: Test your app if it is working or not. If the app is not working, then send an email to the developer with error messages and redeploy the application after the code is being edited by the developer.
Pre-requisites :
- I am using RedHat Enterprise Linux 8 as my base operating system in VirtualBox. You can use any Linux OS to perform this task. In my base OS, I installed two main programs, 1st one is Jenkins and 2nd is Kubectl (Kubernetes Client). I will show you how to set both in a few minutes.
- Next, I am using Minikube in VirtualBox as a Kubernetes server. For creating a local connectivity in my system I created one host-only adapter and in my system, Windows10, RHEL8, & Minikube are connected with it.
- Finally, I am using Git in my Windows system to develop the code and to upload it to GitHub.
Let's set up Jenkins and Kubectl in RHEL8 :
It's very easy to install Jenkins and Kubectl in RHEL8. You just need to run a few commands for each program to setup. I am providing the installation procedure link, just copy those commands and run in terminal. It will install Jenkins and Kubectl program in your system.
- Link to setup Jenkins: https://www.fosstechnix.com/install-jenkins-on-centos-8/
- Link to install Kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/
Great you have successfully install Jenkins and Kubectl. Next, you need to copy 3 files from your Windows OS to RHEL8 to connect the Kubectl with Minikube. Also before doing that install the Minikube and start it.
- For reference check out this link: https://kubernetes.io/docs/tasks/tools/install-minikube/
In windows go to C:\Users\username\.minikube and copy the “ca.crt” file. Next go to C:\Users\username\.minikube\profiles\minikube and copy the “client.crt & client.key” file. Next paste them in the “/root/” folder in the RHEL8. Finally you need to create the configuration file for Kubectl so that it can connect to Minikube. In below you can see the configuration file. The file name should be “config”, be careful and don't give any extension in the name. Next copy this file in the “/root/.kube/” folder and Done.
Great, You are fully set up to jump into the task.
Creation of GitHub repository for Continuous Development of Code :
From GitHub, I created one repository and connect it with my local git command of windows for continuous development of the code and uploading it. Next, I will connect this with Jenkins for Continuous Deployment. Here is the repository I used for this task :
This is a DevOps task so here main focus is on CI/CD automation. So, I haven't focused much on the code. I simply used one html file just to check if my deployment is working fine or not.
Jenkins Job 1 :
Go to jenkins and click on “Manage Jenkins => “Manage Plugins” => available => search for “git” and Install the git plugin.
Next after successful installation click on “New Item” => give name “Job1” => click on Freestyle Project => then ok.
Next, follow the below-mentioned screenshots and put the required information. 1st we are setting up the GitHub repository from where Jenkins will pull the code.
Next I used poll SCM because after I configure it, in each minute Jenkins will check the master branch of the Code and as soon as someone commit then Jenkins will automatically fetch that code.
Also using this Executive shell command I copied all the code which jenkins fetched and store in “/root/DevOpsAL/” location for future use.
One thing to remember is that you need to provide Jenkins the power of sudo, for that either you can edit the “/etc/sudoers” file. Or you can run the below-mentioned command on the terminal.
echo ‘jenkins ALL=(ALL) NOPASSWD:ALL’ >> /etc/sudoersClick on save and then click on “build now”. Next after the build runs let's see the output screen. Click on the build no and then click on Console Output. You can see in the below screen shot that the build runs successfully.
Jenkins Job 2 :
Before starting the Job 2, we need to write one script which will deploy PODs on Kubernetes. As we can see in the problem statement that our data inside PODs needs to be persistent. Also, we need to expose our PODs so that we can do the testing on our code. Now in the below-mentioned YAML file you can find the deployment configuration. If you know how to write Kubernetes deployments configuration then great…, but if you don't know then I suggest
- please go through this link: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
Also this YAML file you have to upload to the same repository where you are developing your code, so that in the future if you want to update the configuration of your deployment then you can easily do that. In the below GitHub repository, you can find that YAML file.
For reference here is the YAML code :
Great, now we are ready to dig into Job 2.
In Job 2, go to “Build Triggers” option and select “Build after other projects are built” and mention Job 1. That means after Job 1 runs successfully, Job 2 will start building automatically. Also check the box, “Trigger only if build is stable”.
Next, we are going to run the kubectl commands to deploy our code. In the code below, you can see the code along with the screenshot.
sudo cd /root/DevOpsAL
sudo ls
if sudo kubectl get all|grep webdeploy
then
sudo kubectl delete all — all
sudo kubectl delete pvc — all
sudo kubectl create -f /root/DevOpsAL/webdeploy.yml
sleep 10
sudo kubectl get all
else
sudo kubectl create -f /root/DevOpsAL/webdeploy.yml
sleep 10
sudo kubectl get all
fi
sudo kubectl cp /root/DevOpsAL/hello.html $(sudo kubectl get pod|grep webdeploy|awk ‘{print$1}’):/var/www/html/hello.htmlGo to Build => Execute shell , and use these commands, remember one thing in your base OS (RHEL8 for me), the folder “/root/DevOpsAL” should be there, otherwise it will fail.
Click on save and then click on “build now”. Next after the build runs let’s see the output screen. You can see in the below screen shot that the build runs successfully.
Great, our deployment is successful. Now just for checking if your website is working fine or not let's go to the Windows Command Prompt to check the IP address of the minikube. In Windows Command Prompt run
minikube service list
Now go to the browser and check if your website is working or not.
Jenkins Job 3 :
So, finally, this is the last Job and this Job will automatically check if the code is running successfully or not. If the code is not running successfully then it will send the developer Email.
To configure the Email Plugin I suggest go through this below-mentioned article
In, Job 3 again select that “Build Trigger” like we did previously and set Job 2 as up stream. In the “Build” select “Execute Shell” and provide these commands mentioned in the below screen shot. This command will check it the website is working fine or not.
Click on save and then click on “build now”. Next after the build runs let’s see the output screen.
Final Work :
Last work is to give this pipeline a beautiful visual UI using Build Pipeline Plugin. So Install the below mentioned plugin.
After Installing the build pipeline plugin, go to the Main Home Page of Jenkins you will find one plus symbol on the top bar of your Job lists. Click on that and then give one View Name, and select Build Pipeline View. Then click on Ok.
Next a page will automatically open, Just check the “Select Initial Job” to your 1st job of your pipeline under “Pipeline Flow”. Next select Ok.
Click on “Run” and you will see your pipeline is working smoothly.
So, that's it, Finally it has been successfully implemented.
Final Words :
- There are lots of Future Possibilities like to do more precise testing using Jenkins, or to deploy more PODs by writing big deployment scripts.
The next part of this Blog You can find in below mentioned link. It's a more advanced way of doing this work.
- I tried to make it as simple as possible. Hope You learned Something from here. Feel free to check out my LinkedIn profile and obviously feel free to comment.
- I write DevOps, Cloud Computing, Machine Learning etc. blogs so feel free to follow me on Medium. And Last but not least if you have any queries DM me on LinkedIn.
