DevOps Task-3 : CI/CD Pipeline Using Jenkins , Kubernetes , GitHub
Kubernetes :
Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.
Why do we need Kubernetes if all our work can be done using docker ?
Ans :- Kubernetes manages the pods running by itself , hence we dont require to do monitoring of pods and do load balancing . It also provides an extra feature of persistent volume through which we can make our data persistent .
Task Description :
1. Create container image that has Jenkins installed using dockerfile .
2. When we launch this image, it should automatically starts Jenkins service in the container.
3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins
4. Job1 : Pull the Github repo automatically when some developers push repo to Github.
5. Job2 : By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed ) .
6. Expose your pod so that testing team could perform the testing on the pod
7. Make the data to remain persistent .
8. Job3 : Test your app if it is working or not.
9. Job4 : If app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer .
NOTE :- Since kubernetes is a container orchestration tool . Hence we don’t have to create a extra Job for monitoring . Kubernetes do monitoring by itself .
Let me first describe “task-3.yml” file through which we will create Service , PV , PVC , Deploment in k8s.
Service :-
A service can be defined as a logical set of pods. It can be defined as an abstraction on the top of the pod which provides a single IP address and DNS name by which pods can be accessed. With Service, it is very easy to manage load balancing configuration. It helps pods to scale very easily.
If you use a Deployment to run your app, it can create and destroy Pods dynamically. Each Pod gets its own IP address, however in a Deployment, the set of Pods running in one moment in time could be different from the set of Pods running that application a moment later. This leads to a problem .
Hence to resolve this problem we create a Service which provides all pods a single IP . The IP will remain same even after all the pods destroy .
# Code
NOTE :- It will expose all the pods having label env : production to port Number 30000 .
Persistent Volume ( PV ):-
Container storage via a container's root file system is ephemeral, and can disappear upon container deletion and creation. To provide a durable location to prevent data from being lost, you can create and use persistent volumes to store data outside of containers.
A persistent volume offers persistent storage that enables your data to remain intact, regardless of whether the containers to which the storage is connected are terminated.
# Code
Here the hostPath is the path of a folder in minikube . I have created hostPath a PersistentVolume .
Minikube :- A tool that runs a single-node Kubernetes cluster in a virtual machine on your personal computer .
Persistent Volume Claim ( PVC ):-
A persistent volume claim (PVC) is a request for storage, which is met by binding the PVC to a persistent volume (PV). A PVC provides an abstraction layer to the underlying storage. For example, an administrator could create a number of static persistent volumes that can later be bound to one or more persistent volume claims. If none of the static persistent volumes match the user’s PVC request, the cluster may attempt to dynamically create a new PV that matches the PVC request.
# Code
Deployment :-
Deployments represent a set of multiple, identical Pods with no unique identities. A Deployment runs multiple replicas of your application and automatically replaces any instances that fail or become unresponsive. In this way, Deployments help ensure that one or more instances of your application are available to serve user requests. Deployments are managed by the Kubernetes Deployment controller .
# Code
I have created 3 replicas of pod for load balancing . By default the strategy used by Deployment is Rolling Update .
Rolling updates allow Deployments update to take place with zero downtime by incrementally updating Pods instances with new ones.
Lastly mounted the above created PVC to the “/usr/local/apache2/htdocs” folder of httpd image , because all the webpages are usually stored in this folder . So we made this folder to be persistent so that whenever the pod terminates due to any reason our data don’t loose .
Now come to hand’s on part
Create container image that has Jenkins installed using dockerfile .When we launch this image, it should automatically starts Jenkins service in the container.
Dockerfile :-
Now to create image from this Dockerfile we have to just run command :-
docker build -t jenkins:latest .Launch a container using this image .
docker run -it — privileged -p 8888:8080 -v /:/host jenkins:latestCreation of jenkins installed image I have explained in detail in Task-2 . So please first go through my Task-2 .
So now directly jump to Jenkins Jobs .
Jenkins :- It is an open source CI/CD tool.
Job1 : Pull the Github repo automatically when some developers push repo to Github.
Used GitHub Webhooks so that whenever a developer pushes code on GitHub , Jenkins Job will automatically trigger .
Shell Script :-
First of all checked whether there is any pre-created directory having same name as “task3-ws” in /host folder . If present then remove it and create a new one . And copy all github files in that .
JOB 2 : By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed ) .
Set Build triggers so that this Job will automatically trigger after successful build of JOB 1 .
Full Code :-
Explaination of above code step by step :-
“ chroot “ :- A chroot environment provides functionality similar to that of a virtual machine, but it is a lighter solution.
1 . So firstly create a new environment using chroot . For detailed explanation of chroot you can go through my Task-2.
2 . Then checked whether the files are of html or php , here I have checked only for html but you can use same approach to check for other extentions also .
3 . Create a new directory “ws-html” particularly for HTML files inside “task3-ws” , if directory already present then firstly removed it and creat a new one. Check all the files and copied only those which are having same extention i.e. “.html” using cmd :-
cp -rvf /task3-ws/*.html /task3-ws/ws-html4 . Now create service , pv , pvc and deployment through “task-3.yml” file using cmd :-
sudo kubectl create -f task-3.yml
5 . And Finally copy all the files present in “ws-html” directory in the minikube directory “/home/docker/devops-task3” using cmd :-
sshpass -p "<minikube_password>" scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -r * docker@192.168.99.101:/home/docker/devops-task3
Console Output :-
JOB 3 : Test your app if it is working or not.
Set Build triggers so that this job automatically runs after successful build of Job 2 .
This Job will check the status code of webpage . If it is 200 then it will terminate with exit code 0 else exit with status code 1 .
NOTE :- 192.168.99.101 is my minikube IP .
Added Post build actions to trigger JOB 4 if this JOB fails .
Console Output :-
My webpage “index.html” :-
JOB 4 : If app is not working , then send email to developer with error messages .
Mail.py file :-
Email Notification if the Job 3 fails :-
Build Pipeline :-
Thanks for reading :-)
