How to setup Jenkins on Kubernetes, the explained way.

TLDR; If you are here you already know what kubernetes is (or I have become incredible famous that clueless people read my posts), anyway, to keep this “tldr”, in this article I’m going to be covering:

• Namespaces
• Deployments
• Services
• PersistentVolumeClaims
• Service Accounts
• Role Bindings

End of the TLDR.

Requirements:

I’ll try to make this as clear as possible, but you’ll need a entry to medium level in the following topics to be able to follow this article:

• Docker
• Kubernetes
• Kubectl
• Jenkins

If you don’t and you still want to dive, do it.

Namespaces

Kubernetes supports multiple virtual clusters backed by the same physical cluster. These virtual clusters are called namespaces.
From the k8 docs.

Human Translation: You can isolate (separate) your resources one from another by placing then in different namespaces. If you think it’s like placing them in two VMs inside the same host machine, I would rather say it is like placing them in different folders on the same machine.

Why would I care? Can’t I just use the default?

Because it will help you separate your resources when you need resources with the same name, or when you only have one cluster but want to setup different environments and you are afraid of overwriting something. Even if you don’t have another cluster to test you could use another namespace.

Yeah you can use the default, but, bear in mind that any changes you made there will affect all your resources without namespaces (except, of course, the ones that just don’t have any).

Enough namespace’s talk, show me your code!

apiVersion: v1
kind: Namespace
metadata:
  name: jenkins-ns

Simple, isn’t it? This will create a namespace called jenkins-ns that we will use thru this “tutorial”.

Deployments

You describe a desired state in a Deployment object, and the Deployment controller changes the actual state to the desired state at a controlled rate.
From the k8 docs.

Human Translation: In this resource you can specify what you need and it will make sure that’s what you get (refering to pods).

I’ll show the code first before further explanation:

This code is not the final deployment code, Hold your horses, you can use it anyway, we will update later

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins
  namespace: jenkins-ns
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
        - name: jenkins
          image: jenkinsci/jenkins
          ports:
            - name: http-port
              containerPort: 8080

This is how you would normally setup your Jenkins deployment, the deployment allows you to specify as much containers per pods as you need, and as much replicas as you need. In case one fails it spins another automagically. In our particular case it will make sure there is 1 and only 1 all the time.

Services

A Kubernetes Service is an abstraction which defines a logical set of Pods and a policy by which to access them - sometimes called a micro-service.
From the k8 docs.

Human translation: Look at this resource as your router rules, under what names the traffic is redirected from the (cluster)network to your application(pod).

This resources requires to match your application/s somehow, and it uses labels to do so.

Remember we had this on our deployment?

metadata:
      labels:
        app: jenkins

This is exactly what our service will look at (metaphorically speaking) to find our app.

apiVersion: v1
kind: Service
metadata:
  name: jenkins-svc
  namespace: jenkins-ns
spec:
  type: NodePort
  ports:
    - name: http
      port: 8080
      targetPort: 8080
  selector:
    app: jenkins

What is going on in here? We had defined the service, we told it that its name is “jenkins-svc”, and it will be in the jenkins-ns’s namespace, that it will take the traffic that reach the port 8080 and redirect it to the 8080 defined on our pod inside the deployment, and what pods is what I’m talking about? whichever has the app: jenkins assigned.

Persistent Volume And Persistent Volume Claims

A PersistentVolume (PV) is a piece of storage in the cluster that has been provisioned by an administrator.
From the k8 docs.

Human translation: A PV is basically a resource that the cluster admins create, that allows them to specify how they will divide the space.

A PersistentVolumeClaim (PVC) is a request for storage by a user. It is similar to a pod. Pods consume node resources and PVCs consume PV resources.
From the k8 docs.

Human translation: A PVC is basically a resource that allows whoever is creating the pod to request one of those segmented storages (PVs).

Trick: Depending on the cluster config the PVC can generate the PV, like the Deployment generates a replica set, all under the hood. Easy!.
__That’s what I’ll exactly do, for “convenience”.__

Be careful with the following command, it will take 70Gi out of your HDD, feel free to reduce that size.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pvc
  namespace: jenkins-ns
  labels:
    app: jenkins
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 70Gi

So what is happening in here? nothing new, we define the resource type(kind) we give it a name, tell it to be saved under the jenkins-ns, we give it a label, and the interesting part is, as always, on the spec. We defined the access mode as ReadWriteOnce that means it can only be accessed by one pod at a time. We also set the resources.requests storage to be 70Gi.

This creates a PV with the default driver, the specified size and the access mode.

Service Account

A service account provides an identity for processes that run in a Pod.
From the k8 docs.

The most clear description they’ve ever made.

No so necessary, but I’ll give it to you for the sake of consistency, human translation: This resource allow you to give your resources an identity that can then be used to authenticate it (give or remove access from your resources), you know security and stuff.

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: jenkins-ns
  name: jenkins-sa

Simply, net, short.

What is going on here, where is all that magical config you promised us? On another resource, the rules itself doesn’t go in here.

this leads us to our last resource before I can show you the full deployment config, which might be the entire reason you are here.

Role bindings

A role binding grants the permissions defined in a role to a user or set of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. Permissions can be granted within a namespace with a RoleBinding, or cluster-wide with a ClusterRoleBinding.
from the k8 docs.

A even less necessary, human translation: The role binding is where you specify what permissions a. resource have where.

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins-role-binding
subjects:
  - kind: ServiceAccount
    name: jenkins-sa
    namespace: jenkins-ns
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

This is one of the trickier things on kubernetes but, I hope, at this point you should understand it clearly.

What is going on here? We have defined the kind as ClusterRoleBinding, we have given it a name, a specified the subjects we said that we want to link a ServiceAccount by it’s, previously defined, name jenkins-sa on the namespace jenkins-ns and that the role it will take is, under roleRef, of kind ClusterRole , name cluster-admin and of the same api group as the ClusterRoleBinding .

Something that I must specified in here, instead of giving it cluster-admin access you should give it specific access to the resources because security an stuff, also any pod that matches those specs will be turn into a cluster admin and will be able to access, and destroy, your cluster. More security and stuff >:).

After all the resources have been created we can now process to reconfigure our deployment to make good use of these.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: jenkins-ns # limit it to the jenkins-ns namespace (ns)
  name: jenkins
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      serviceAccountName: jenkins-sa # Here we use the SA
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts # Change it for what you want
          env:
            - name: JAVA_OPTS
              value: -Djenkins.install.runSetupWizard=false
          ports:
            - name: http-port
              containerPort: 8080
          volumeMounts:
            - name: jenkins-pvc
              mountPath: /var/jenkins_home
      volumes:
        - name: jenkins-pvc
          persistentVolumeClaim:
            claimName: jenkins-pvc # call the PVCs

What is going on here? We used the jenkins-ns namespace, therefore everything defined here will go there, we use the serviceAccountName so it uses that service account to connect to the other cluster resources, we pass an env var telling it to skip the initial setup and send us directly to the home screen, we then specify the volumes and tell it to use the jenkins-pvc, and that jenkins-pvc is a reference to the pvc we created in the previous steps.

I had plans for a part 2 including how to setup the master/slave flow on kubernetes-jenkins. ;)