CICD with K3S and GitLab

Background

While working on a project, we needed tools to deploy our code to production or development environment. CICD is a mechanism for helping you deploying to host with your own pipeline mechanism that include test, build, deploy or more you can add many steps to comply with business requirements.

Installing K3S

K3s is a lightweight Kubernetes distribution that is very easy to use and lightweight. Only a simple installation script is needed to install K3s to the host.

First you need to check the minimum requirement for installing K3s, check the link below.

https://docs.k3s.io/installation/requirements

# curl -sfL https://get.k3s.io | sh - 
[INFO] Finding release for channel stable
[INFO] Using v1. 29.3 +k3s1 as release
[INFO] Downloading hash https: //gi thub.com /k3s-io/ k3s /releases/ download /v1.29.3+k3s1/ sha256sum-amd64.txt
[INFO] Downloading binary https: //gi thub.com /k3s-io/ k3s /releases/ download /v1.29.3+k3s1/ k3s
[INFO] Verifying binary download
...
...
[INFO] systemd: Starting k3s

• The K3s service will be configured to automatically restart after node reboots or if the process crashes or is killed
• Additional utilities will be installed, including kubectl, crictl, ctr, k3s-killall.sh, and k3s-uninstall.sh
• A kubeconfig file will be written to /etc/rancher/k3s/k3s.yaml and the kubectl installed by K3s will automatically use it

Then, try kubectl command and make sure status are ready.

$ kubectl get nodes
NAME         STATUS   ROLES                  AGE   VERSION
my-vps1   Ready    control-plane,master   9d    v1.29.3+k3s1

If you have permission problem when running kubectl command try to run this command

$ sudo chown $USER:$USER /etc/rancher/k3s/k3s.yaml

Gitlab Runner

https://docs.gitlab.com/runner/install/kubernetes.html

First create the gitlab runner configuration file.

# config.yaml
gitlabUrl: https://gitlab.com/
concurrent: 2
rbac:
  create: true

runners:
  secret: gitlab-runner-secret
  config: |
    [[runners]]
      [runners.kubernetes]
        image = "ubuntu:22.04"
        privileged = true
      [[runners.kubernetes.volumes.empty_dir]]
        name = "docker-certs"
        mount_path = "/certs/client"
        medium = "Memory"

  ## Specify the name for the runner.
  name: "gitlab-runner"

Then build your token in your repository or projects, in this example will cover runner within your project. to get your token go to side bar Build -> Runners -> New Group Runner -> Create Runner

creating group runner in gitlab

Don’t forget to add tags, it will be used on .gitlab-ci.yml later.

# gitlab-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-runner-secret
type: Opaque
data:
  runner-registration-token: "" # your token from gitlab in base64
  runner-token: "" # your token gitlab in base64

After creating the configuration for gitlab runner, install helm to the instance before installing gitlab runner into K3s.

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
$ helm version
version.BuildInfo{Version:"v3.14.4", GitCommit:"81c902a123462fd4052bc5e9aa9c513c4c8fc142", GitTreeState:"clean", GoVersion:"go1.21.9"}

-- applying secrets into K3s environment
$ kubectl apply -f gitlab-secret.yaml

-- installing gitlab runner to K3s
$ helm install --namespace default gitlab-runner -f config.yaml gitlab/gitlab-runner

-- if you want updating configuration
$ helm upgrade --install --namespace default gitlab-runner -f config.yaml gitlab/gitlab-runner

Make sure your runner already online in your gitlab dashboard. after that we can add .gitlab-ci.yml to our project and see it runs.

stages: # List of stages for jobs, and their order of execution
  - test
  - build
  - deploy

unit-test-job: # This job runs in the test stage.
  image: golang:1.22.2-alpine3.19
  tags:
    - "tutor"
  stage: test # It only starts when the job in the build stage completes successfully.
  before_script:
    - apk add --no-cache make && apk add --no-cache bash
    - wget -O- -nv https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.1
    - golangci-lint --version
    - go install github.com/uudashr/gocognit/cmd/gocognit@latest
    - go version
    - go get -v -d ./...
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - make test

build:
  variables:
    IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG-$CI_COMMIT_SHORT_SHA
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"
    DOCKER_TLS_VERIFY: 1
    DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client"
  tags:
    - "tutor"
  image: docker:24.0.5
  services:
    - docker:24.0.5-dind
  stage: build
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG

deploy-job: # This job runs in the deploy stage.
  image:
    name: bitnami/kubectl:latest
    entrypoint: [""]
  tags:
    - "tutor"
  stage: deploy # It only runs when *both* jobs in the test stage complete successfully.
  environment: production
  script:
    - echo "Deploying application..."
    - echo "Migrate..."
    - echo "Application successfully deployed."
    - kubectl config get-contexts
    - kubectl get pods

This configuration we are creating 3 stages pipelines test, build and deploy. For each stage you can do multiple jobs, for example in test stage we can just add another job within test stage.

...
sonarlint:
  tags:
    - "tutor"
  stage: test # see here
  script:
    - echo "Running unit tests... This will take about 60 seconds."

Voila! its done, hope it helps..