Deploy Applications on Kubernetes Cluster with GitLab CI/CD Tunnel

Published in

Linux Shots

5 min readFeb 6, 2022

CI/CD is used for quick and efficient delivery of an application to its consumers. This enables quick bug fixes, frequent security updates of an application, increases the speed of innovations and make application available to the consumer in lesser time.

Today, Kubernetes is widely being used to deploy microservices and containerized applications. GitLab has a feature under Infrastructure Management which allows us to connect our Kubernetes Clusters with GitLab. This enable us to deploy, manage and update application running on Kubernetes Cluster with CI/CD tunnel without exposing Kubernetes APIs to Internet. This is available in GitLab 14.5 and above for free and GitLab 13.11 and above for premium users.

Here, We will learn how to connect an existing Kubernetes Cluster to GitLab and How to deploy applications on cluster using CI/CD tunnel. Stay tuned.

Pre-requisites

This demo is performed on/with:

GitLab (Gitlab.com) v14.8.0-pre (Self-managed GitLab can also be used)
Kubernetes Cluster v1.21.5 (Running locally / K8s API endpoint unexposed to internet)
GitLab Agent for Kubernetes
Ubuntu 20.04 as Bastion
Docker 20.10.12 on Bastion

Connect Kubernetes Cluster to GitLab

To connect Kubernetes Cluster to GitLab, We need to create a separate repository which will be used to store agent configuration and connect to GitLab Agent which will be running in cluster.

If you are using GitLab self-managed instance, You need to enable GitLab Kubernetes Agent Server (KAS) from GitLab configuration using below step.

Add below line in /etc/gitlab/gitlab.rb file

gitlab_kas['enable'] = true

Restart or re-configure GitLab.

Generate a Personal Access Token to login to GitLab registry.

Go to Preferences → Access Tokens
Give name of token and expiry date. Select scope as read_registry (and write registry if you need that for any other use. Not required for this demo.)
Copy the generated PAT and keep it handy.

2. Login to GitLab registry on bastion. Give your GitLab username and generated PAT in password.

docker login registry.gitlab.com

3. Create a repository in GitLab to store agent configurations. (Repo A)

Create an empty repository.
Add configuration file .gitlab/agents/<agentname>/config.yaml

You can choose any name for agent. In my case I have chosen ‘mygitlabagent’. And my repo name is ‘kubernetes-cluster’. My agent configuration file will be .gitlab/agents/mygitlabagent/config.yaml

4. Click on Infrastructure → Kubernetes Cluster option from left side panel.

5. Click Actions button on top right and Select the agent name from drop down option. Click Register.

6. A registration token will be generated. Copy that token and save it somewhere safe for future use (Once window is closed, token can not be viewed).

7. A command will also be generated under Recommended Installation Method section. Copy that command and run on your bastion. This will generate a manifest and deploy GitLab agents in your K8s cluster. Agent pod will be deployed in gitlab-kubernetes-agent namespace.

8. Wait for agent pod to come up. Refresh the GitLab page and you should be able to see Connection Status of agent as ‘connected’.

Create application repository

We will create repository where our application manifests will be stored.

Create your application repository (Repo B).
Add K8s manifest yamls of your application in the repository.
Create a file .gitlab-ci.yml. Refer below example CI to deploy application with kubectl. Explanation added in comments in the content.

#--------- GitLab Agent CI/CD Tunnel configuration# Set KUBE_CONTEXT to use with the GitLab Kubernetes Agent's CI/CD tunnel. See# https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_tunnel.html.#-----------------------------------# This will get access of your K8s cluster from GitLab Agent using its service account.variables:
  # KUBE_CONTEXT: usernameorgroupname/agentconfigprojectname:agentnameKUBE_CONTEXT: linuxshots/kubernetes-cluster:mygitlabagent# Set K8s context of GitLab Agent.kube-context:
  before_script:
  - if [ -n "$KUBE_CONTEXT" ]; then kubectl config use-context "$KUBE_CONTEXT"; fi# Deploy application on K8s cluster using kubectldeploy:
  extends: [.kube-context]
  rules:
  - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # CI will run when any new commit is made to default branch (main)image:
    name: bitnami/kubectl:latest
    entrypoint: [""]
  script:
  - ls
  - kubectl apply -f $CI_PROJECT_DIR/sampleapps/. # Directory where yamls are placed in repo
  - kubectl get pods -n sampleapp

Authorize application repository to access GitLab Agent

Above created application repository (Repo B) cannot use CI/CD tunnel using GitLab agent unless it is authorized to do so. Any GitLab project or group can be authorized to access GitLab agent by adding configuration in agent configuration repository (Repo A).

Add below contents in agent configuration repository (Repo A) in file .gitlab/agents/<agentname>/config.yaml

ci_access:
  projects:
  # - id: usernameorgroupname/repositoryname
  - id: linuxshots/k8s-applications # Project/Repo name to authorize
  
  # groups:
  # - id: /path/to/group/groupname # To authorize a group

You can add any new repository or group when you need to authorize it.

Once above configuration is added, Any new commit in default branch of application repository will trigger the pipeline and deploy your K8s resources defined in YAMLs.

Deploy using helmfile

In above steps, we used kubectl to deploy the resources on cluster. We can also use helm chart to deploy and update the application.

Add a file in application repository’s root directory helmfile.yaml. Below example is to deploy Hashicorp vault using its helm chart.

# Helm repository information
repositories:
- name: vault-helm
  url: https://helm.releases.hashicorp.com# Helm release information.
releases:
- name: vault # Release name
  namespace: vault # Namespace in which chart to be deployed
  chart: vault-helm/vault  # Chart name
  version: 0.5.0 # Chart version
  installed: true
  values:
  - values.yaml # values.yaml file to override default values.

Add a values.yaml file in root directory and add the keys and values which you want to override from default values. Example below for vault.

ui:
  enabled: true

Add .gitlab-ci.yml file with helm deployment stage. Refer below example:

variables:
  KUBE_CONTEXT: linuxshots/kubernetes-cluster:mygitlabagent.kube-context:
  before_script:
  - if [ -n "$KUBE_CONTEXT" ]; then kubectl config use-context "$KUBE_CONTEXT"; fideploy:
  extends: [.kube-context]
  stage: deploy
  image: "registry.gitlab.com/gitlab-org/cluster-integration/cluster-applications:v1.3.2" # This image can be used to deploy helmfile
  environment:
    name: production  script:
  - gl-ensure-namespace vault # Ensures if namespace exists, if not it creates it create
  - gl-helmfile --file $CI_PROJECT_DIR/helmfile.yaml apply --suppress-secrets  rules:
  - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Once committed to default branch, Pipeline will be triggered and helmfile will be deployed.

For more details on how to use Kubernetes Cluster using GitLab, follow GitLab official document from here.

Thank You

Navratan Lal Gupta