Highly scalable Jenkins with kubernetes

Introduction:

This blog is going to visit how to build highly scalable Jenkins with multiple master and dynamic nodes on kubernetes.

Pre-requisite for Jenkins to be installed, you need a working kubernetes cluster, you can visit my previous blogs on how to build kubernetes with KOPS in AWS , if you plan to run this workload in production follow the blog Production grade K8 setup with KOPS.

Jenkins is a popular CI tool for building and packaging code, and sometimes its even used for deploying code to various environments. Jenkins architecture is based on classic Master, agent/node configuration. The Jenkins Master receives all build requests, and delegates the workload to execute on the agent/nodes, but it keeps the metadata like job no, logs, history, and configurations on the master. Jenkins master writes all the metadata and the configuration to a filesystem. Since K8 is not inherently stateful, ie if the container is stopped, it doesnt retain state and data is wipedout, hence for jenkins master to retain data we need to implement data persistency.

Persistent Volumes and Stateful sets:

In K8 recently stateful sets (pet sets previously) has been introduced. Statefulsets maintains sticky identity for the Jenkins master pod. Additionally we need to implement persistent volumes and there are several ways to achieve it via rook, ceph, glusterfs, nfs, EBS etc.. I would suggest to refer my previous blog to setup persistent storage for your K8 cluster Persistent storage using glusterfs.

Architecture:

We will be creating an infrastructure to host multiple Jenkins masters as shown below. The Jenkins Master will be a docker instance connected to a persistent volume. We will be using the kubernetes jenkins plugin to dynamically create the Jenkins agent to execute the workloads. A new pod will be scheduled on the K8 node to host Jenkins agent when a new workload becomes available, after the workload is executed the pod and hence the Jenkins agent will be destroyed and resources will be freed up. The various dev teams can access their own Jenkins instance via a single load balancer identified by an unique context root.

Installation and Configuration:

1. Create a K8 cluster for POC or for Production workloads
2. Setup persistent storage using glusterfs

Coming soon …

1. Deploy Jenkins Master
2. Install kubernetes jenkins plugin
3. Create a test job
4. Validate the job