GitLab CI/CD with Kubernetes

9 min readNov 16, 2022

GitLab is an open-source CI/CD platform that is a solid alternative to Jenkins. GitLab is also a preferred remote git repository tool after GitHub acquisition by Microsoft.

Being a remote git server with the functionality to provide a complete CI/CD solution is what attracts the majority of people to switch to GitLab. GitLab CI/CD script is written in YAML.

Create GitLab Account: -

Getting started with GitLab is also easy, first, we need to create an account in GitLab,

Go to GitLab.com and then click sign up or simply go to this URL https://gitlab.com/users/sign_up
Fill out a simple form containing the basic information about the user who is creating the account.
There will be a confirmation email on your provided email account confirming your account by visiting that link and you are done!!!

After this you would need a GitLab runner to use the CI/CD part of the GitLab, you can install GitLab runner on any Linux operating system (Ubuntu, RedHat, Fedora and Centos).
It also works on your local machine as well as docker and any public cloud (AWS, AZURE, Digital-Ocean).
GitLab runner has a free tier that limits you to 400 CI/CD minutes.

The GitLab-runner can work in any cloud which offers a Compute Machine, in our case we use Digital-Ocean droplets to create or set up our GitLab-runner.

Follow this link to create a Digital-Ocean account: -

https://cloud.digitalocean.com/registrations/new

Create Droplet (Compute Machine/Virtual Machine) For GitLab-Runner: -

After this create a droplet by clicking on create which can be located in the top right corner after you log in to your account, make sure that you give minimum hardware requirements to the droplet to run the GitLab runner.

Minimum hardware requirements to run GitLab-runner: -

RAM — 2GB RAM is the recommended memory size.
CPU — 2 cores are the recommended number of cores.
Disk Storage — minimum 25 GB of disk space.

Follow these steps in a sequential manner to create a Droplet (Virtual Machine): -

Create, you can find this on the top right corner
Droplets, select this option to build the virtual machine
Choose Region, select the region which is near to you or your target audience.
Choose an operating system Image
Choose a Plan (Basic, General Purpose, CPU Optimized, Memory Optimized, Storage Optimized)
Generate ssh keys or Password to access the machine

Steps to install GitLab-runner on droplet

Considering you selected the default Debian server,

Just follow these commands to install GitLab-runner

sudo apt update
sudo apt upgrade
sudo apt install GitLab-runner

Now to verify this installation run the following command

GitLab-runner

It should output the normal help option.

Setup a Kubernetes cluster

Now we need to setup a Kubernetes cluster in which we can deploy our build/code.

To setup a Kubernetes cluster, I highly recommend you use any public cloud provider (AWS, AZURE, DIGITAL OCEAN) which provides the Kubernetes service, in our case, we will use the Digital-Ocean Kubernetes service.

To create the Kubernetes cluster, just click create in the top right corner the same way you did to create a droplet(In Digital Ocean Cloud).

Follow these steps in a sequential manner to create the Kubernetes cluster

Click on Create
Choose a data center (select which is near to you or your target audience).
Select a version (go with the default here).
Choose your pool capacity (for this example minimum configuration will work)
Select 1 node instead of the default 3.
Skip to the step where you name your cluster, provide the desired name
Check your monthly costs to be double-sure about your configuration.
Click on create a cluster.

After the installation and making of the GitLab account, you are ready to run your first pipeline.

In this example, we will follow a three-step process to deploy a Kubernetes pod to cluster

We created a docker file to build docker images for applications.
Push the image to the docker hub/container registry (if using digital ocean)
Push the pod to the cluster which will make use of the above docker image.

Now we need to configure our GitLab account to run the CI/CD pipeline.

Initial setup of GitLab account to run the CI/CD -

Variables in the pipeline are one of the most important parts to keep things separate and avoid exposing the password in the pipeline so we make variables to achieve this.
GitLab supports the use of variables, by creating an environment which is to keep the variables separate to run in different phases of development.

To create an environment, follow the steps in a sequential manner -

After you logged in to the GitLab account, in the left pane you will find the operations option
Hover over the operations option and there will be a sub-option called environment.
Click on the environment option.
Create an environment by ingesting a name to it.

To create variables, follow the steps in a sequential manner -

In the left pane, you will find an option setting.
Hover over the setting option, click on the CI/CD option that will show up
Find variables on the CI/CD option page.
Now create the variable by clicking on add variable option.

Now as we are done setting variables and environments now, we have to set up the GitLab-runner in the GitLab account.

Note: -

Configuration of GitLab runner as we are doing it for the very first time, this is a one-time setup.

after this, you just have to a basic setup to activate the GitLab runner on your repository,

Download the binary for your system

sudo curl -L — output /usr/local/bin/GitLab-runner https://GitLab-runner-downloads.s3.amazonaws.com/latest/binaries/GitLab-runner-linux-amd64

Give it permission to execute

sudo chmod +x /usr/local/bin/GitLab-runner

Create a GitLab CI user

sudo useradd — comment ‘GitLab Runner’ — create-home GitLab-runner — shell /bin/bash

Install and run as service

sudo GitLab-runner install — user=GitLab-runner — working-directory=/home/GitLab-runner

sudo GitLab-runner start

After this you need to register your GitLab-runner by following command

sudo GitLab-runner register — url https://GitLab.bigohtech.com/ — registration-token $REGISTRATION_TOKEN

Fill in the basic obvious details which will be asked after this command to get your GitLab runner setup

There will be 2 main things in the configuration -

First your registration token
Second the name you will give to your GitLab runner

With the setup of the environment, variables, and GitLab-runner you are ready to build your first CI/CD pipeline

Now go to the source files in your repository and add a file and name it .gitlab-ci.yaml this file will tell GitLab to follow these steps to run your pipeline.

Creating a very basic CI/CD pipeline to deploy our Kubernetes pods.

stages:

- deploy

Deploy_Dev:

stage: deploy

environment:

name: <k8s> (environment variable name you defined)

when: manual

before_script:

- ls -l

- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY_URL

script:

- docker build -t $CI_IMAGE_NAME .

- docker tag $CI_IMAGE_NAME $CI_REGISTRY_IMAGE_URL:$IMAGE_TAG

- docker push $CI_REGISTRY_IMAGE_URL:$IMAGE_TAG

- cat “$DEPLOYMENT_FILE” > deployment.yaml

- cat “$KUBE_CONFIG_FILE” > k8s-config.yaml

- export KUBECONFIG=k8s-config.yaml

- kubectl apply -f deployment.yaml

tags:

- k8s (GitLab-runner name you defined)

Explanation of the script above -

First, we defined a stage name.
Then we define a custom name for our build in our case we used Deploy_Dev.
Trigger jobs will be run manually.
Then before going for deployment, we log in to our docker service which is located on variables on the setting of CI/CD.
After this, we go for the build.
First, we built a docker image and then push this image to the DigitalOcean container registry.

The pushed image will look like this:

A common failed pipeline error

In this image looking closely we forgot to specify the location of Dockerfile that’s why this pipeline failed.

To counter this error keep checking about the typos you make, check whether the command is right or not.

Reference to common pipeline error -

https://about.gitlab.com/blog/2022/03/08/how-to-troubleshoot-a-gitlab-pipeline-failure/

Docker image push error

If the image is not pushed properly, it won’t be shown in the docker hub or any other container image storage service.

To solve check whether the docker file is in the correct format and performing the action correctly which is mentioned, if not change it to the correct one.

Refer to this link for common docker errors -

https://docs.docker.com/desktop/troubleshoot/topics/

Accessing- pods:-

After this, we get our Kubernetes deployment and kube-config file which is also on variable file on the setting of CI/CD.

then we deploy on a cluster with kubectl apply command.

Your pod will look something like this if it’s running properly.

What if the pod is not in the running state, there could be two types of errors now

Image pull back off
Back off restart

ImagePullBackOff occurs when the pod cannot fetch the image, make sure to add the secret file to get the secret of the docker hub/container registry service

Example image to show how the image pulls back off looks like after we run
kubectl describe pod pod_name

Back-off restart error occurs for various reasons, one of the main reasons is the docker file is not written correctly.

Refer to this link for more such common issues -

https://opendatascience.com/common-issues-with-kubernetes-deployments-and-how-to-fix-them/

Common Kubernetes error -

Some common errors include the error kubectl command not found.

This error will occur even after you export the Kube config file, as the command is still dependent to be installed separately.

To fix this run -

sudo apt update
sudo apt upgrade
sudo apt install kubectl

Common Kubernetes commands you could use for the debugging of your pod

Export your Kubernetes configuration file location depending on the cloud you choose to use, for reference we are giving you links for some public cloud.

After this, you need some commands to interact with the cluster.

kubectl get pods (this will list all the pod in the default namespace)
kubectl get svc (this will list all the services)
kubectl get ns (this will list all the namespace)
kubectl describe pod *pod_name* (this will open the show what is happening inside the pod)
kubectl logs *pod_name* (this will show the logs of the pod which you mentioned)

Follow these steps to manually trigger a pipeline

On the left pane, go to the option CI/CD and
Then click on the pipeline option you will see a screen where your CI/CD pipeline trigger will live.
Click on the play button and now you can run the pipeline manually.