CI/CD pipeline to deploy Nginx to a k3s cluster
The created action will deploy a Nginx web page to k3s with GitHub Actions. There are many different tools to do something similar but this is what I used.
- Kubernetes cluster — Linode is a cheap cloud option
- GitHub account (Install Git as well)
- Docker Hub account (Install Docker)
- Tail Scale account( On PC, and cluster)
If you are replicating this project bear in mind that many how-to articles quickly become outdated. It is rare that instructions are able to be translated with a 1:1 ratio. In time this may become a broad overview. I am by no means an expert but everything I have learned so far has been from solving problems. So although your project may look a bit different this should help you understand the basic deployment process.
Actions
What really made Actions click for me was understanding its basic purpose. Actions is just another computer executing remote scripts. So to do that, you just need a way to connect securely and run them in the right order. For a larger more enterprise setup Actions may send jobs to more services that allow more complicated deployments to multiple environments.
Security
One of the most important parts of CI/CD and software development is safely securing credentials, secrets, and keys. GitHub has an integrated secrets manager that you can securely save all of this.
Getting Started
Create a repo with these files
The cornerstone of this project is GitHub Actions. The main workflow files are stored in .github/workflows. The workflow file is called CI-CD.yml for this project. This is the start of a basic workflow, all other jobs will be added after this part.
name: Deploy to K3s
on:
push:
branches:
- main
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
After you have the initial repo setup and add a workflow file you must get credentials and add them to the secrets in GitHub.
Docker Hub
Used as a container repository.
Obtain the access tokens from Docker Hub by navigating to “My account”->”Security”
Create a Docker file that points to the files.
FROM nginx:latest
COPY index.html /usr/share/nginx/html/index.html
The Docker portion of the workflow sets up build, logs in, and then updates the docker image.
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Build and Push Docker Image
run: docker buildx build --platform linux/amd64 -t ${{ secrets.DOCKER_HUB_USERNAME }}/nginx-custom:latest --push .
Tailscale
Tailscale is used to create a secure connection between authorized devices.
“Settings"‐>”OAuth clients"
curl -fsSL https://tailscale.com/install.sh | sh
I was able to install Tailscale on all of my nodes with the above command. After running this command you will be prompted to visit a web link and log in.
The Tailscale portion makes a connection possible between the Kubernetes API server and GitHub Actions.
- name: Setup Tailscale
uses: tailscale/github-action@v2
with:
oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
tags: tag:ci
Kube Config
cat ~/.kube/config | base64
Cat your Kube Config and add it to GitHub. The Kube Config must be in base64 format to work properly when stored as a secret. The only change required is changing the server IP to the Tailscale IP assigned to your Kubernetes API server.
This section configures Kubectl to use the Tailscale IP address of your cluster.
- name: Set up Kubectl
run: |
mkdir ${HOME}/.kube
echo ${{ secrets.KUBECONFIG }} | base64 --decode > ${HOME}/.kube/config
Deployment files
Kubectl is used to apply these two Yaml files and then restart the deployment. For a production environment you should probably make sure you have a secure tls connection. I was not really worried about it for a dev environment.
This uses kubectl to apply the files below. One sets specs for the deployment and the other sets up a Nodeport to access it. Then it redeploys changes to the cluster.
//actually a nodeport
- name: Deploy loadBalancer
run: |
kubectl apply -f .github/nb-cd.yml -n nginx --insecure-skip-tls-verify
- name: Deploy Nginx
run: |
kubectl apply -f .github/ng-cd.yml -n nginx --insecure-skip-tls-verify
- name: Redeploy Changes
run: |
kubectl rollout restart deployment/nginx-deployment -n nginx --insecure-skip-tls-verify
Access deployment
After deploying this you will be able to access the web page at any node IP, and the corresponding port. You can also connect with the Tailscale IP(from another machine with Tailscale).
Although this is very simple you can easily add more website files to the html directory and deploy a complete static site. The next project I write about will be showing a CI/CD pipeline for Go services that I integrate with a Nginx website.
All files
nb-cd.yml
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: nginx
spec:
type: NodePort
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
ng-cd.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: yourusername/nginx-custom:latest
ports:
- containerPort: 80
resources:
limits:
cpu: "0.3"
memory: "512Mi"
requests:
cpu: "0.1"
memory: "128Mi"
tolerations:
- key: "k3s-controlplane"
operator: "Equal"
value: "true"
effect: "NoExecute"
CI-CD.yml
name: Deploy to K3s
on:
push:
branches:
- main
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Setup Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Build and Push Docker Image
run: docker buildx build --platform linux/amd64 -t ${{ secrets.DOCKER_HUB_USERNAME }}/nginx-custom:latest --push .
- name: Setup Tailscale
uses: tailscale/github-action@v2
with:
oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
tags: tag:ci
- name: Set up Kubectl
run: |
mkdir ${HOME}/.kube
echo ${{ secrets.KUBECONFIG }} | base64 --decode > ${HOME}/.kube/config
//actually a nodeport
- name: Deploy loadBalancer
run: |
kubectl apply -f .github/nb-cd.yml -n nginx --insecure-skip-tls-verify
- name: Deploy Nginx
run: |
kubectl apply -f .github/ng-cd.yml -n nginx --insecure-skip-tls-verify
- name: Redeploy Changes
run: |
kubectl rollout restart deployment/nginx-deployment -n nginx --insecure-skip-tls-verify
Dockerfile
FROM nginx:latest
COPY index.html /usr/share/nginx/html/index.html
Index.html
<!DOCTYPE html>
<html>
<head>
<title>Example</title>
</head>
<body>
<p>This is a website!</p>
</body>
</html>