Pulumi, GCP, and Gitlab Runners

I have recently been getting really into Pulumi, and I am finding it help my life in really cool ways. For example, I have a gitlab runner that I create from a GCP Compute Instance.

Pulumi has allowed me to automate the creation and deletion of these runners with incredible ease.

I won’t bore you with getting started with Pulumi. There are many excellent articles on that, and I don’t have anything to add, at this time. (However, I will eventually write an article on how I manage a large project with it)

Startup

Before I get into Pulumi code, I want to show the Compute Instance startup-script and shutdown-script. This way, you will have an understanding of the steps they use to build the instances.

I am starting from a raw Debian image, and using the startup script to completely set up the runner. I could create an image, but that won’t help with this explanation.

Comments explain steps. Note the [BRACKETS] which denote values you may want to replace. You should also read through commands carefully. They may need modification to meet your specific needs. See the commands respective documentation for clarity.

# obligatory initial update
sudo apt update
# Install docker
sudo apt install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
sudo apt update
sudo apt install -y docker-ce
# Download the gitlab runner binary for your system
sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
# Give it permission to execute
sudo chmod +x /usr/local/bin/gitlab-runner
# Create a GitLab Runner user
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
usermod -aG docker gitlab-runner
# restart docker
sudo service docker restart
# use gitlab-runner to install and run it's service
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start
# Register your runner
sudo su - gitlab-runner
gitlab-runner register \
  --name [INSERT THE RUNNER NAME] \
  --url https://gitlab.com/ \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
  --docker-volumes /root/.docker/config.json:/root/.docker/config.json:ro \
  --registration-token [INSERT YOUR REGISTRATION TOKEN HERE] \
  --docker-privileged \
  --executor "docker"  \
  --docker-image docker:20.10.16 \
  --tag-list [ADD TAGS HERE]\
  --run-untagged="false"  \
  --locked="false" \
  -n
# Optional Step - Set your Google Service Account so your docker instance can get images from Google Artifact Registry
echo "[SERVICE ACCOUNT JSON, BASE64 ENCODED]" | tee key.txt
cat key.txt | base64 -d | docker login -u _json_key --password-stdin https://us-central1-docker.pkg.dev
# This will add a cron to nuke old docker containers occasionally. VERY helpful in keeping your runner clean. 
echo "0 12  * * 2 /usr/bin/docker system prune -a --volumes -f" > /var/spool/cron/crontabs/root

Shutdown

Likewise, the shutdown-script works to de-provision our instance.

# Let's cleanup the runner from gitlab
gitlab-runner unregister --name [INSERT THE RUNNER NAME]

Putting it all together

I use Python with Pulumi, so you will need to adjust this code to whatever language you chose to use with Pulumi. This is the gcp.compute.Instance definition for the runner. It’s the only definition you should need to get it up and running.

gitlab_runner_be = gcp.compute.Instance(runner_name,
  name=runner_name,
  project=project,
  boot_disk=gcp.compute.InstanceBootDiskArgs(
    device_name="persistent-disk-0",
    initialize_params=gcp.compute.InstanceBootDiskInitializeParamsArgs(
      image="debian-cloud/debian-11",
      size=100,
    ),
  ),
  allow_stopping_for_update=True,
  machine_type="n1-standard-2",
  metadata={
    "shutdown-script": f"""
# Un register runner
gitlab-runner unregister --name {runner_name}
""",
  "startup-script": f"""
# obligatory initial update
sudo apt update
# Install docker
sudo apt install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
sudo apt update
sudo apt install -y docker-ce
# Download the gitlab runner binary for your system
sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
# Give it permission to execute
sudo chmod +x /usr/local/bin/gitlab-runner
# Create a GitLab Runner user
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
usermod -aG docker gitlab-runner
# restart docker
sudo service docker restart
# use gitlab-runner to install and run it's service
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start
# Register your runner
sudo su - gitlab-runner
gitlab-runner register \
  --name [INSERT THE RUNNER NAME] \
  --url https://gitlab.com/ \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
  --docker-volumes /root/.docker/config.json:/root/.docker/config.json:ro \
  --registration-token [INSERT YOUR REGISTRATION TOKEN HERE] \
  --docker-privileged \
  --executor "docker"  \
  --docker-image docker:20.10.16 \
  --tag-list [ADD TAGS HERE]\
  --run-untagged="false"  \
  --locked="false" \
  -n
# Optional Step - Set your Google Service Account so your docker instance can get images from Google Artifact Registry
echo "[SERVICE ACCOUNT JSON, BASE64 ENCODED]" | tee key.txt
cat key.txt | base64 -d | docker login -u _json_key --password-stdin https://us-central1-docker.pkg.dev
# This will add a cron to nuke old docker containers occasionally. VERY helpful in keeping your runner clean. 
echo "0 12  * * 2 /usr/bin/docker system prune -a --volumes -f" > /var/spool/cron/crontabs/root
""",
  },
  network_interfaces=[gcp.compute.InstanceNetworkInterfaceArgs(
  access_configs[gcp.compute.InstanceNetworkInterfaceAccessConfigArgs()],
  network="default",)],
  service_account=gcp.compute.InstanceServiceAccountArgs(
    email=service_account_email,
    scopes=["cloud-platform"],
  ),
  zone=zone,
opts=pulumi.ResourceOptions(protect=False))

This is a mouthful, but hopefully you can see how it all works.

This isn’t everything that I have added to the startup script, and you may want to add other things too. For instance if you have a kubernetes cluster that your CI/CD deploys to, you may want to configure your cluster information in the startup script. Or if you have other weird dependencies.

Happy Ops’ing!