CI/CD pipeline to deploy Nginx to a k3s cluster

The created action will deploy a Nginx web page to k3s with GitHub Actions. There are many different tools to do something similar but this is what I used.

• Kubernetes cluster — Linode is a cheap cloud option
• GitHub account (Install Git as well)
• Docker Hub account (Install Docker)
• Tail Scale account( On PC, and cluster)

If you are replicating this project bear in mind that many how-to articles quickly become outdated. It is rare that instructions are able to be translated with a 1:1 ratio. In time this may become a broad overview. I am by no means an expert but everything I have learned so far has been from solving problems. So although your project may look a bit different this should help you understand the basic deployment process.

Actions

What really made Actions click for me was understanding its basic purpose. Actions is just another computer executing remote scripts. So to do that, you just need a way to connect securely and run them in the right order. For a larger more enterprise setup Actions may send jobs to more services that allow more complicated deployments to multiple environments.

Security

One of the most important parts of CI/CD and software development is safely securing credentials, secrets, and keys. GitHub has an integrated secrets manager that you can securely save all of this.

Getting Started

Create a repo with these files

The cornerstone of this project is GitHub Actions. The main workflow files are stored in .github/workflows. The workflow file is called CI-CD.yml for this project. This is the start of a basic workflow, all other jobs will be added after this part.

name: Deploy to K3s

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2

After you have the initial repo setup and add a workflow file you must get credentials and add them to the secrets in GitHub.

Docker Hub

Used as a container repository.

Docker Hub

Obtain the access tokens from Docker Hub by navigating to “My account”->”Security”

Create a Docker file that points to the files.

FROM nginx:latest
COPY index.html /usr/share/nginx/html/index.html

The Docker portion of the workflow sets up build, logs in, and then updates the docker image.

- name: Setup Docker Buildx
      uses: docker/setup-buildx-action@v1

    - name: Login to DockerHub
      uses: docker/login-action@v1
      with:
        username: ${{ secrets.DOCKER_HUB_USERNAME }}
        password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
    
    - name: Build and Push Docker Image
      run: docker buildx build --platform linux/amd64 -t ${{ secrets.DOCKER_HUB_USERNAME }}/nginx-custom:latest --push .

Tailscale

Tailscale is used to create a secure connection between authorized devices.

“Settings"‐>”OAuth clients"

TailScale web console

curl -fsSL https://tailscale.com/install.sh | sh

I was able to install Tailscale on all of my nodes with the above command. After running this command you will be prompted to visit a web link and log in.

The Tailscale portion makes a connection possible between the Kubernetes API server and GitHub Actions.

- name: Setup Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci

Kube Config

cat ~/.kube/config | base64

Cat your Kube Config and add it to GitHub. The Kube Config must be in base64 format to work properly when stored as a secret. The only change required is changing the server IP to the Tailscale IP assigned to your Kubernetes API server.

This section configures Kubectl to use the Tailscale IP address of your cluster.

- name: Set up Kubectl
      run: |
        mkdir ${HOME}/.kube
        echo ${{ secrets.KUBECONFIG }} | base64 --decode > ${HOME}/.kube/config

Deployment files

Kubectl is used to apply these two Yaml files and then restart the deployment. For a production environment you should probably make sure you have a secure tls connection. I was not really worried about it for a dev environment.

This uses kubectl to apply the files below. One sets specs for the deployment and the other sets up a Nodeport to access it. Then it redeploys changes to the cluster.

 //actually a nodeport 
    - name: Deploy loadBalancer 
      run: |
        kubectl apply -f .github/nb-cd.yml -n nginx --insecure-skip-tls-verify    

    - name: Deploy Nginx
      run: |
        kubectl apply -f .github/ng-cd.yml -n nginx --insecure-skip-tls-verify

    - name: Redeploy Changes
      run: |
        kubectl rollout restart deployment/nginx-deployment -n nginx --insecure-skip-tls-verify

Access deployment

After deploying this you will be able to access the web page at any node IP, and the corresponding port. You can also connect with the Tailscale IP(from another machine with Tailscale).

Although this is very simple you can easily add more website files to the html directory and deploy a complete static site. The next project I write about will be showing a CI/CD pipeline for Go services that I integrate with a Nginx website.

All files

nb-cd.yml
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: nginx
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

ng-cd.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: nginx  
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: yourusername/nginx-custom:latest
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: "0.3"
            memory: "512Mi"
          requests:
            cpu: "0.1"
            memory: "128Mi"
      tolerations:
      - key: "k3s-controlplane"
        operator: "Equal"
        value: "true"
        effect: "NoExecute"

CI-CD.yml
name: Deploy to K3s

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2

    - name: Setup Docker Buildx
      uses: docker/setup-buildx-action@v1

    - name: Login to DockerHub
      uses: docker/login-action@v1
      with:
        username: ${{ secrets.DOCKER_HUB_USERNAME }}
        password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
    
    - name: Build and Push Docker Image
      run: docker buildx build --platform linux/amd64 -t ${{ secrets.DOCKER_HUB_USERNAME }}/nginx-custom:latest --push .


    - name: Setup Tailscale
      uses: tailscale/github-action@v2
      with:
        oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
        oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
        tags: tag:ci

    - name: Set up Kubectl
      run: |
        mkdir ${HOME}/.kube
        echo ${{ secrets.KUBECONFIG }} | base64 --decode > ${HOME}/.kube/config
      
      //actually a nodeport 
    - name: Deploy loadBalancer 
      run: |
        kubectl apply -f .github/nb-cd.yml -n nginx --insecure-skip-tls-verify    

    - name: Deploy Nginx
      run: |
        kubectl apply -f .github/ng-cd.yml -n nginx --insecure-skip-tls-verify


    - name: Redeploy Changes
      run: |
        kubectl rollout restart deployment/nginx-deployment -n nginx --insecure-skip-tls-verify

Dockerfile

FROM nginx:latest
COPY index.html /usr/share/nginx/html/index.html

Index.html

<!DOCTYPE html>
<html>
    <head>
        <title>Example</title>
    </head>
    <body>
        <p>This is a website!</p>
    </body>
</html>