How we built a GitHub App to level up our CI pipeline
This is the 5th edition of the Breathe Life Engineering Blog. This month’s article is written by Pascal Giguère.
In just a decade, GitHub didn’t only grow to become the biggest code hoster in the world, it introduced many features to software developers that completely changed the way they think about programming.
We at Breathe Life have been using GitHub’s collaborative features extensively for all our software projects. In the past few months, we’ve also been experimenting with GitHub Apps, a relatively new feature aimed at improving the way GitHub repositories integrate with external services.
What’s a GitHub App, anyway?
Introduced only a few months ago, GitHub Apps are now the officially recommended way to integrate with GitHub. Developers have the option to either develop private apps to use them in their own projects, or to make them publicly available on the GitHub Marketplace for anyone to install.
The Marketplace might make you think of mobile apps, such as the ones offered on the App Store or Google Play. However, at the technical level, installing a GitHub App has very little in common with installing a smartphone app. You can think of it as giving permission for an external service to interact with your repository. As such, GitHub Apps only need to be updated when they require a permission change, and feature updates generally happen without the user’s intervention.
Prior to introducing GitHub Apps, GitHub offered an API for what they called Commit Statuses, little green checkmark / red X mark icons. These statuses would appear next to each commit after some tasks in external services (usually triggered through Webhooks) have been run. GitHub Apps now offer Check Runs, a more advanced and flexible take on the same basic concept, as one of their main features.
Using Google’s Cloud Build GitHub App
When GitHub introduced Apps late last year, they were proud to name Google as one of the major developers of a continuous integration (CI) app on the GitHub Marketplace. The Google Cloud Build app allows Google Cloud Platform users to automatically trigger cloud builds whenever a new commit is pushed to their GitHub repository.
As existing Google Cloud Build users ourselves, we were immediately interested in using the official GitHub App to strengthen our CI pipeline. It enabled us to run builds, unit tests and end-to-end tests more frequently, allowed us to catch errors earlier in our workflow, and ultimately improved confidence in our code. We’ve found the Cloud Build app to be a great addition to our toolset. Over time, however, we’ve grown to understand its limitations and lack of flexibility.
Where the Cloud Build app fell short
If you’ve used Cloud Build triggers in the past, you’ve probably come to appreciate the level of flexibility they offer. You can create build triggers that target only some git branches, use different Cloud Build manifest files for different triggers, and inject substitution variables that alter each build.
Unfortunately, none of those exist within the Cloud Build GitHub App. You provide a single manifest file (A cloudbuild.yaml at the root of your repo), which may not be injected with substitution variables, and builds will be triggered for all pushes to all branches.
At Breathe Life, we store our different software projects inside a single large git monorepo. Each software product can be built for different partners (life insurance carriers), providing a unique branding and feature set for each partner. So for example, if we have 2 products that can each be built for 3 partners, that’s 6 different builds we can make.
The main issue we had with the Cloud Build app was that when new commits were pushed, it couldn’t figure out which of our 6 projects changed and which corresponding build had to be triggered. Since all projects potentially changed, that means we’d have to trigger all 6 builds on every push to reliably spot bugs.
Since the Cloud Build app also only supported one build per commit, we’d have to combine the 6 builds together by creating a single long Cloud Build manifest file describing build steps for all of our projects. With each project taking around 10 minutes to build, we’d be up to a full hour to complete all 6. And as we scale up our number of partners, we’d be looking at hours of build time for every single push to any our monorepo’s branches.
This approach simply couldn’t scale. We’d clog up Cloud Build’s queue way faster than builds could finish. To keep build times reasonable, we had to be able to trigger a build only for a certain product and a certain partner, rather than every possible combination of them, something you can’t do with the Cloud Build GitHub App.
How we built our own GitHub App
The good news was that the APIs used by the Cloud Build GitHub App are all available to the general public. Theoretically, we could then build our own GitHub App that replicates the functionality of the Cloud Build app, only with more flexibility regarding what is built and when. And that’s precisely what we decided to do.
The first step to creating a GitHub App was to register it. This can be done from GitHub’s developer settings page. From there, you can choose the app’s name, icon, permissions, and whether you want to publish it to the GitHub Marketplace. Once registered, the app will have an App ID, a Client ID and a Client secret associated to it.
The second step was to install our GitHub app to our GitHub repository. This is achieved by visiting the “Install App” section of your app’s settings. Once installed, you’ll be given an Installation ID.
Then, we configured GitHub Webhooks for the specific events we were interested in. Those Webhook events are what will call our app to trigger builds. You can set them up in the “Webhooks” section of your repository’s settings.
Now that we were all set up, we needed to actually write the app’s code. We chose to build a Node app written in TypeScript and deployed to Google Cloud Functions, but there is no restriction on your choice of language, framework or infrastructure. The only requirement is to be able to interact with the GitHub REST API.
The trickiest part of using GitHub’s API was dealing with authentication. In order for our app to interact with the GitHub API, we must authenticate twice. First, the GitHub App itself must authenticate using its App ID, Client ID and Client secret obtained in the first step. And second, the app installation must also authenticate, using the Installation ID from the second step. Luckily, the unofficial GitHub API Node client octokit made that job easier for us.
“Cloud Build on PRs”, our Google Cloud Build app alternative
We chose Cloud Build on PRs as the name for our custom GitHub App. As the name implies, its job is to trigger cloud builds following pull request events. Unlike the official Cloud Build app, builds aren’t triggered for every push to every branch, they’re only triggered when a PR is opened, or if new commits are pushed to that PR’s branch (what GitHub calls a PR synchronization).
But more importantly, what sets Cloud Build on PRs apart from the Cloud Build app is its ability to trigger different types of cloud builds depending on which files were edited as part of the PR. For example, if the edited files’ paths only indicate changes in product1 for partnerB, only that specific build will be triggered. This can be achieved by using different manifest files and inserting substitutions variables, two important features that the Cloud Build app lacks.
We provide Cloud Build on PRs with a config file that defines the different types of builds that can be triggered.
Each build type has a label, which will appear in GitHub’s Check Runs list, a corresponding manifest file, and a list of paths to watch for changes in order to trigger the build. The same path can be watched by different build types, perfect for files that are shared across different projects.
Cloud Build on PRs gave us the same continuous improvement advantage promised by Google’s Cloud Build app: being able to spot bugs directly from within GitHub before new PRs are merged. Only, its “build only what’s needed” approach made it a viable option for us even though we use a large monorepo for numerous different projects.
Want to learn more about how GitHub Apps are implemented? We’ve recently open-sourced Cloud Build on PRs, so feel free to peek into its code if you’re looking for an example — or fork it entirely if you’re looking to replicate its functionality for your own repository.
