Photo by Roman Synkevych on Unsplash

GitHub Actions self-hosted runners with AWS CDK

Cost-optimized, elastic, and fully managed, on AWS 🚀

GitHub Actions, GitHub’s own CI/CD solution, is pretty sweet in that it’s fully integrated with the usual GitHub user experience, resulting in tremendous amounts of soft values and synergies, especially for those who spend all their awake time on GitHub.

So what do you do if you are, for example, on GitHub Enterprise Server and there are no Actions runners available or if you need more execution minutes than what’s included in your GitHub plan?

You launch your own runners!

Right, so how does that work?
The most basic setup is quite easy, it’s just a matter of configuring and spinning up a binary that registers with GitHub. However, for things to scale and work well in a bigger organization there has to be a process around everything which takes care of stuff like scaling, performance, security, and cost-efficiency.

This is where AWS CDK comes in.
We have produced a higher-level CDK construct library to make it easy, fun, and convenient to provision GitHub Actions workers in a scaleable way, on AWS, without any fuzz.

👉 https://github.com/schibsted/github-actions-self-hosted-cdk 👈

Because it’s a pre-baked CDK library, anyone can start using it today and enjoy their self-hosted GitHub Actions runners, but because this is a blog post, let’s dig into how it works, what’s going on under the hood, and how it has been designed.

So, let’s get to it!

The way the CDK construct works is that, when it’s being provisioned, with cdk deploy, it sets up everything needed to have a full-blown fully event-driven GitHub Actions environment. That is:

• It produces an AMI, an Amazon Machine Image, with all the bells and whistles needed to run the GitHub Actions runner with Docker support.
• It also deploys a Lambda-backed API Gateway, responsible for launching new EC2 instances when jobs are queued in GitHub Actions, based on the pre-baked AMI.
• The usual suspects. Networking, and security groups.

How it works under the hood

The Lambda listens to webhook events from GitHub and launches new short-lived EC2 instances as needed.

• Which instance type to use is defined on the GitHub Actions job level and can vary throughout a workflow, making it possible to use beefier instances for heavy workloads and smaller instances for jobs that don’t require much resources.
• The EC2 instances can be configured to either be Spot instances (i.e. possibly cheaper) or use on-demand pricing.
• All instances are set up to self-destruct when their job is completed or after some pre-defined timeout, both for cleanup purposes and to prevent unexpected recurring costs.
• Networking can be configured to be either private or public. Private means that instances are launched in a private subnet and all outgoing traffic is routed through a NAT Gateway with a static IP. Public means public subnets, but don’t worry, the instances are still protected with restrictive security groups.
• By being strict about using ephemeral instances we can guarantee that each and every job get a clean and isolated environment to run in.

Boxes and arrows

This drawing visualizes how all the things tie together and relate to each other.

Solution architecture

Alrighty, how do get this off the ground?

Here’s when AWS CDK really comes in to shine.
Because a CDK library is nothing but an NPM package it’s trivial to distribute and start using, just like you would use any other NPM package.

(AWS CDK works with several different programming languages and one does not necessarily have to use Typescript, but it’s by far the most popular choice so we’ll stick to that today)

There’s a better and more detailed guide on how to get started in the READMEbut here’s the gist of it:

First things first (getting ready for CDK)

1. Make sure your ~/.aws/credentials is properly set up.
2. Install the CDK CLI, npm install -g aws-cdk.
3. Bootstrap your AWS account for CDK by running by following the instructions in the CDK Bootstrapping guide.

Create a new CDK project

1. Create a new Typescript project.
2. Add@schibsted/github-actions-self-hosted-cdk and aws-cdk-libas dependencies.
3. Shove this into a file and adjust to your likings:

Off we go! 🚀

1. Provision everything with cdk deploy.
2. Configure GitHub to send workflow_job events to the endpoint that is printed by the end of the cdk deploy run.

That’s it. Reasonably straightforward.
You now have a full-blown event-driven self-hosted GitHub Actions environment ready waiting for you in your AWS account!

Yay, cool, anything else?

Yeah, a certain label is required on every workflow job to specify which instance type to use. It can be any (x86 based) instance type available in the region where this is running.

This example shows a workflow where the build jobs are running on m5.medium instances and the deploy jobs on t3.micro.

What’s next?

At the top of the nice-to-have-we-should-totally-add-this list, we have support for ARM/Graviton2 based EC2 instances. This is likely to come soon-ish.

Thanks for reading, cheers and have a good day! 🍻

PS: We’re hiring and have exciting positions in all our locations across the Nordics and Poland. Check out our open positions at https://schibsted.com/career/.