Google Compute Engine CI/CD with GitHub Actions
Deploy a Docker image to Google Compute Engine by simply ‘git push’-ing.
I made the following to be able to update my website from my command line. Basically continuous deployment. In its final state, it should really only update when my main branch is updated but for now it updates with any new commit.
Thousand Foot View aka Thousand Foot Run-On Sentence
First, a GitHub Action waits for a git-push event to kick-off a workflow that follows an instructional YAML file running on a GitHub-hosted VM to build an image that follows a Dockerfile and deploy the image to Google Compute Engine to run a container that executes a command to run your Node.js application.
Assumptions
- This guide assumes you can already ‘git push’ to GitHub.
- Also that you are using a custom domain name from Google Domains.
Set up your repository
Create the repository on GitHub
Log into GitHub and navigate to https://github.com/new to create a new repository.
Create the repository.
Put the repository on your computer
Follow the instructions on the webpage or navigate in the command line to where you would like your project folder to be on your computer and run:
git clone <https URL from the Quick setup section>
‘git clone’ will create a folder that is linked to the repository for when you push to GitHub (i.e. ‘git push’).
Create a feature branch and push it to remote (while setting its upstream branch).
git checkout -b cloud-deploy
git push --set-upstream origin cloud-deploy
This will contain the rest of the work of this tutorial. Eventually you will merge into the ‘main’ branch when you are done.
Set up your Google Cloud project
Log into Google and navigate to: https://console.cloud.google.com/projectcreate.
Pick a project name and press “CREATE” to create your Google Cloud Project. It may take a few seconds for it to be created.
Enable the APIs
Use either the left-hand Navigation Menu or the top search bar to find and enable the Container Registry and Compute Engine APIs.
Create the virtual machine
Create a new Container-Optimized VM instance via the “CREATE INSTANCE” button in the top navigation or via the middle button (only displayed if no VMs exist).
To keep this project within the free tier , select the following options:
Region: us-west1, us-central1, or us-east1.
Machine configuration: General-purpose > E2 > e2-micro.
Boot disk: Boot disk type > Standard persistent disk.
Check the box under “Container” for “Deploy a container image to this VM instance.”
We have not yet published an image so do not worry about filling that field in with a real value. For now use, “gcr.io/google-containers/busybox”.
Under “Volume mounts” add a volume mount like so:
Volume Type: Directory
Mount path: /etc/letsencrypt
Host path: /home/<your domain name>/etc/letsencrypt
Mode: Read/write
Then two others like so:
Volume Type: Directory
Mount path: /var/lib/letsencrypt
Host path: /home/<your domain name>/var/lib/letsencrypt
Mode: Read/writeVolume Type: Directory
Mount path: /var/log/letsencrypt
Host path: /home/<your domain name>/var/log/letsencrypt
Mode: Read/write
This allows us to persist the files that the container is writing to these folders onto the host system. We don’t map /etc on the container to /etc on the host because the /etc folder is not saved when the VM restarts (same for /var). I do not follow the convention that folders inside /home are user folders but I found that this folder persisted between restarts and chose this to keep it organized.
Change the boot disk OS to a Container Optimized OS.
Allow HTTP/HTTPS traffic so we can talk to the VM in the later steps.
The other options can be left as default.
For completeness, here is the setup I am using:
Create a new service account
Navigate to https://console.cloud.google.com/iam-admin/serviceaccounts and click the “CREATE SERVICE ACCOUNT” button in the top navigation bar.
Name the service account and then click “CREATE AND CONTINUE” button.
Grant permissions to your service account via assigning roles that allows it to manage VMs (Compute Instance Admin (beta)), push images to the Container Registry (Compute Storage Admin), and run as the service account user (Service Account User). These permissions are overly permissive and should be adjusted in a true production environment. Once you are up and running are start using the service account, Google will show you which ones you are actually using. From there you can adjust down to these specific permissions. This can be done on the Permissions page under IAM & Admin > IAM.
You do not need to “Grant users access to this service account” in the listed 3rd step. After completing the 2nd step, click “DONE”.
Reminder: To change service account permissions, the page to do so is confusingly the “IAM” page instead of the “Service Accounts” page. Go figure.
Generate a service account key
Next you will need to create a service account key. You will give this key to GitHub Actions as a secret so that it can use the service account you just created.
Navigate to the “Service Accounts” page then click on the service account you just created. Under the “KEYS” tab, click on “ADD KEY” then “Create new key”.
Choose “JSON” as the key type and click “CREATE”.
The JSON file will save to your desktop. Keep this file secure as you would a password as this key gives access that you set above.
Add the service account key to your repository’s secrets
Navigate to your repository on GitHub and add a new repository secret via Settings > Secrets > Actions secrets > “New repository secret” button.
Name the secret GCE_SA_KEY and copy the contents of the JSON file wholesale as the value.
Your custom secrets are not copied when repositories are forked. Your secrets are safe with GitHub! https://docs.github.com/en/actions/reference/encrypted-secrets
Add another secret for GCE_PROJECT that is your Google Cloud project ID. This information can be found under your project dashboard. I don’t believe that this information is sensitive and thus needs to be a secret. It may just be the workflow using it as a variable instead of hardcoding it.
Set up your local project and Dockerfile
Install Node.js.
In your working directory on your computer, run ‘npm init -y’. The ‘-y’ flag automatically accepts the default parameters (e.g. project name).
‘npm init’ initializes your package.json file which is what will be used when building your application.
Next use npm to install Express.js and Socket.io. We will use Express.js as the web framework and Socket.io to establish connections for a latency test. Run the following command:
npm install express socket.io
In your project’s root folder create a folder “public” with an html file inside “index.html” with some basic boilerplate and a latency test.
<!DOCTYPE html>
<html lang="en">
<head>
<title>Hello, world!</title>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<meta name="description" content="" />
</head>
<body>
<h1>Hello, world!</h1><p>Latency is:</p>
<p id="latency">INITIALIZING</p>
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io(); setInterval(() => {
const start = Date.now(); socket.volatile.emit("ping", () => {
const latency = Date.now() - start;
console.log(latency + " ms"); const latencyText = document.querySelector('#latency');
latencyText.innerText = latency + " ms";
});
}, 5000);
</script>
</body>
</html>
In your project’s root folder create two more files, “test-app-http.js” and “test-app-https.js”.
In “test-app-http.js” put:
const express = require('express');
const path = require('path');
const app = express();app.use(express.static(path.join(__dirname, 'public')));app.get('/', function (req, res) {
res.sendFile(path.join(__dirname, 'public', 'index.html'));
});app.listen(80);
This will serve ‘index.html’ at port 80 and make anything inside the “public” folder accessible (needed for later when proving ownership for SSL certs with the http-challenge).
In “test-app-https.js” put:
const express = require('express');
const fs = require('fs');
const path = require('path');
const app = express();
const https = require('https');
const { Server } = require("socket.io");const privateKey = fs.readFileSync('/etc/letsencrypt/live/<your domain name here>/privkey.pem');
const certificate = fs.readFileSync('/etc/letsencrypt/live/<your domain name here>/fullchain.pem');const credentials = {
key: privateKey,
cert: certificate
}const httpsServer = https.createServer(credentials, app);const io = new Server();
io.attach(httpsServer);app.use(express.static(path.join(__dirname, 'build')));app.get('/', (req, res) => {
res.sendFile(path.join(__dirname, 'build', 'index.html'));
});io.on('connection', (socket) => {
console.log('a user connected');
});httpsServer.listen(443, () => {
console.log('listening on *:443');
});// Start latency test
io.on("connection", (socket) => {
socket.on("ping", (cb) => {
if (typeof cb === "function")
cb();
});
});
// End latency test
This does the same as “test-app-http.js” but over an encrypted connection using HTTPS instead of HTTP plus a latency test at the bottom.
The latency test could have been in the “test-app-http.js” file as well but I wanted to keep that one simple and limit its function to just what’s required for the http-challenge that we will need later.
Create a Dockerfile
Next, create a file in your project’s root folder called “Dockerfile” (Note: no file extension) with the following contents:
FROM node:14
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
RUN apt-get update && apt-get install -y cron certbot
CMD certbot certonly --webroot --agree-tos --email <your email> -d <your domain name> -w public --keep-until-expiring --no-eff-email --pre-hook "node test-app-http.js" --post-hook "node test-app-https.js" --test-cert && node test-app-https.jsThe Docker image is being built from a base node image.
Then the package*.json files are copied. Copying just package*.json first allows us to utilized cached layers when building the image locally (no effect on the GitHub Action runners). More information here: https://nodejs.org/en/docs/guides/nodejs-docker-webapp/
Next “npm install” is run (use “npm ci” in a production environment) and the rest of the image is built (installing cron and certbot along the way).
Running “apt-get update” and “apt-get install” together ensures that they form an image layer together. Had they been separated, it’s possible that “apt-get update” could have made a layer that is much older (from an earlier cache) which is then used in apt-get install.
Once built, when it is run, it executes the last CMD command. This command is to allow automated SSL certificate issuance and renewal from Let’s Encrypt. The pre-hook is to run the http server so that we can prove that we own the listed domain (-d flag in the certbot command). The post-hook is so that when renewals are needed and completed, it starts the server.
certbot creates a twice-daily cron job to check for renewals and uses the same options as the original certbot command
I haven’t decided if I want a command to shut down the server in the pre-hook so this node command in the post-hook is mostly a placeholder. More information on certbot’s command-line options here: https://certbot.eff.org/docs/using.html#certbot-command-line-options
Create a .dockerignore file
Create a new file in the same folder at your Dockerfile called “.dockerignore” with the following contents:
node_modules
npm-debug.logConfigure your GitHub Action
Create a new file in your local IDE at the following location:
.github/workflows/deploy-to-gce.yaml
Copy the entire contents of this example workflow into this new file.
Update values for your project
Change “master” to “main” under which branch pushes should cause the workflow to run. Also temporarily add the current branch that you’re on for testing. Earlier, I had named mine “cloud-deploy”.
Update the GCE_INSTANCE and GCE_INSTANCE_ZONE values with those for your VM. Values can be found under Compute Engine > VM instances.
What is happening in the rest of the workflow?
For the full docs on workflow terminology check here: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-action. Otherwise, continue below for specifics.
The first section details the jobs and which platform to run the job on. In this case, it’s using the latest ubuntu build.
The first step of this job checkouts your code. For each usage of “uses”, you can see the actual code that’s being run if you navigate to that repo. For example:
action/checkout@v2
-> https://github.com/actions/checkout
or
google-github-actions/setup-gcloud@master
-> https://github.com/google-github-actions/setup-gcloud
The “@” denotes a specific version, branch, commit, or tag.
The next step of the job configures the runner to use your credentials when executing gcloud commands in the subsequent steps. It also configures gcloud to use the project you specified in the secrets earlier.
The next 4 steps use the “run” keyword. This runs the specified commands in the operating system’s shell.
Note: $GITHUB_SHA is an automatically generated variable that is the commit SHA for the commit that kicks off the workflow. We use it as a unique identifier for the image that we push to the Google Container Registry.
Add additional steps to your workflow
Before your Deploy step add:
# Add pruning and IP address update to VM startup script
- name: Update startup script to prune and update IP address
run: |-
gcloud compute instances add-metadata $GCE_INSTANCE \
--zone "$GCE_INSTANCE_ZONE" \
--metadata=startup-script="#! /bin/bash
docker image prune -af
curl --location --request GET 'domains.google.com/nic/update?hostname=<your domain name here>' \
--header 'User-Agent: VM' \
--header 'Authorization: Basic $DDNS_AUTH_STRING'"This updates a property on your VM called “startup-script” that is run once the VM has been initialized. In this case, you are pruning unused images off of the VM (your container is running by the time this script is run so no worries about being unable to start your Docker container due to a missing image).
This is required due to finite space requirements on your VM. Without a pruning step, eventually your VM would be unable to download the latest image (since all the old images would be there) and the VM would fail to start with the new image.
The cURL command being run is hitting an API endpoint that is specific to Google Domains (my registrar) to update the IP address that the domain name is pointing to (it takes the IP of the computer that is making the request). The auth is a username/password combination that has been base64 encoded and uploaded to GitHub as a repository secret just like the other repository secrets.
After your Deploy step add:
# Purge old images from GCR (not latest)
- name: Purge GCR images
run: |-
gcloud container images list-tags gcr.io/$PROJECT_ID/$GCE_INSTANCE-image \
--format="get(digest)" --filter="NOT tags=$GITHUB_SHA" | \
awk -v image_path="gcr.io/$PROJECT_ID/$GCE_INSTANCE-image@" '{print image_path $1}' | \
xargs -r gcloud container images delete --force-delete-tags --quietThis will delete the images that are hosted on Google Container Registry so that you aren’t pushing so many images that you exceed the Free-Tier limit. It searches for images that aren’t the one that it just pushed and deletes those ones.
The End
Make sure it works and merge into main. Then change the workflow to only trigger on merges to main so that production deployments don’t happen with every development commit.
Turn off billing or delete your project for financial peace of mind.
Extra Credit
- Add a billing kill-switch: https://github.com/aioverlords/Google-Cloud-Platform-Killswitch
- Use GitHub Actions with the built-in scheduler and the certbot docker image instead of installing cron & certbot
Sources
https://socket.io/get-started/chat
https://github.com/google-github-actions/setup-gcloud/tree/master/example-workflows/gce
