How to manually install and configure GitLab Runners on GKE (Google Kubernetes Engine)
When you use GitLab CI/CD, you have access to GitLab Shared Runners for running your jobs. You have absolutely no configuration to do, it’s ready to use.
But Shared Runners have some disadvantages. They are limited in build time (2000 minutes/month on Free Plan), you can’t configure them and they can be pretty slow…
That’s why GitLab lets us host our own runners, they can be hosted wherever you want, locally on your machine, on a server, in a Docker container…
In this tutorial, we will see how to host our runners on Kubernetes and particularly on Google Kubernetes Engine. At Ovrsea, we chose Kubernetes because of its capability to autoscale, when we have a lot of jobs running we want to increase the amount of runners available automatically and in the other hand, when we have no jobs running, we don’t wan’t to pay for it.
There is an option to automatically create a Kubernetes cluster on GKE (Google Kubernetes Engine) on the GitLab interface but this is not what this article is about because it’s somewhat limited and it can be tricky to configure afterwards.
Cluster Creation
First, you have to create a cluster on GKE.
In the Node Pools options, you have to choose your machine type, I personally chose n1-standard-2 (2 vCPUs and 7.5GB memory) and only 1 node.
Click on “More options” to set “Autoscaling” on and choose minimum and maximum nodes you want. This allows us to scale our nodes when there are too many pods in one node (one job in our CI/CD is one pod in a Kubernetes node).
If you are not comfortable with Kubernetes autoscaling, I suggest you to read this article https://dzone.com/articles/kubernetes-autoscaling-explained.
The rest of configuration is up to you, I personally let the default settings.
Helm and Tiller Installation
We will install Helm which is a package manager for Kubernetes, and Tiller which is the server who will communicate between Helm and Kubernetes.
First, get a connection to our newly created cluster by clicking “Connect” in the clusters view.
You should now have a terminal view in the bottom of the page with a prefilled line, just hit “Enter”.
Download Helm:
wget https://storage.googleapis.com/kubernetes-helm/helm-v2.6.2-linux-amd64.tar.gzExtract it:
$ tar zxfv helm-v2.6.2-linux-amd64.tar.gz$ cp linux-amd64/helm .
Set authorizations for Tiller:
$ kubectl create serviceaccount --namespace kube-system tiller$ kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller$ kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
Install Tiller:
$ ./helm repo add gitlab https://charts.gitlab.io$ ./helm init
Configure your values.yml
Now, create a values.yml file, it will contain all the configuration for the GitLab Runners.
$ touch values.yml$ nano values.yml
You can see at https://gitlab.com/charts/gitlab-runner/blob/master/values.yaml all available variables but I will cover the most important ones.
# values.ymlgitlabUrl: https://gitlab.com/
runnerRegistrationToken: <your-GitLab-Runners-Registration-Token>
concurrent: 20
rbac:
create: true
clusterWideAccess: false
runners:
image: ubuntu:18.04
privileged: false
cache:
cacheType: s3
cacheShared: true
s3ServerAddress: s3.amazonaws.com
s3BucketName: <your-S3-bucket>
s3BucketLocation: us-central-1
s3CacheInsecure: false
secretName: s3access
builds:
memoryRequests: 4000Mi
gitlabUrl: Don’t change this, it always be https://gitlab.com/
runnerRegistrationToken: You can get this token in your CI/CD settings on GitLab, in the Runners section.
concurrent: The number of concurrent jobs you can run at the same time.
rbac: For RBAC support, don’t change it.
runners: Configuration for deploying the runners, see below.
image: Default container image.
privileged: For executing docker commands in the containers, don’t change it as we don’t need it.
cache: As jobs will run in different pods and nodes, if you use cache for saving dependencies between jobs, you have to save it somewhere else. You can use either GCS on Google Cloud Platform or S3 on AWS. I personally use S3. You will need a secret file filled with your credentials, you can create it with the following commands:
For S3:
$ kubectl -n <your-cluster-name> \
create secret generic s3access \
--from-literal=accesskey="YourAccessKey" \
--from-literal=secretkey="YourSecretKey"For GCS:
$ kubectl -n <your-cluster-name> \
create secret generic gcsaccess \
--from-literal=gcs-access-id="YourAccessID" \
--from-literal=gcs-private-key="YourPrivateKey"
builds: Configuration for the builds, you can specify here how much memory and CPU your jobs need. If a node is out of memory or CPU usage, the cluster will create another node for you, thanks to autoscaling.
You can now apply this configuration to your cluster:
$ ./helm install --namespace <your-cluster-name> --name gitlab-runner -f values.yml gitlab/gitlab-runnerTadaa ! Your runner is now ready to use !
If you need to change the current configuration, you just have to edit your values.yml and upgrade your runner with the following command:
$ ./helm upgrade --namespace <your-cluster-name> -f values.yml gitlab-runner gitlab/gitlab-runnerLink GitLab with your runner
GitLab should now be linked to your runner if you specified the good registration token in your values.yml file.
You can now trigger your jobs and see pods in action in the GKE Workloads menu.
That’s it, you should have a working GitLab Runners environment by now. If you are facing some issues, feel free to ask me in comments.
For further reading on Kubernetes and GitLab Runners:
