How to set up merge queues in GitHub Actions

GitHub Merge Queues is a cool feature for repositories that have a long-running CI and a lot of pull requests requiring that CI.

GitHub talks about that feature at length in their blog post, but the gist of it is that it allows you to run CI actions on multiple PRs at the same time stacked on top of each other. This allows you to remove the annoying “upmerge main” step you otherwise continuously need to do when other people’s pull requests are merged before yours.

Unfortunately, the documentation for how to integrate GitHub Merge Queues with GitHub Actions is not entirely straightforward and misses some details you’ll be scrambling to figure out on your own. Luckily, I already did, so I can present it to you in this explainer-slash-tutorial.

The explainer-slash-tutorial comes in two parts: explainer and tutorial.

Explainer: how does CI for Merge Queues work

The merge queues are based on two other features of GitHub you’ll need to understand first: branch protection rules and automerging of pull requests.

Branch protection rules allow you to force a certain set of ceremonies around merging to your branch. For example, you can forbid directly pushing to main or force a CI run before merging. If you are looking into merge queues, you probably have both of those already enabled.

The merge queues themselves are also a form of branch protection so this is a new checkbox you’ll need to enable in that interface.

Automerging is a feature on top of the protection rules. It allows you to press the “Merge” button before the CI is finished, and the merge will be performed when/if it’s successful.

Merge queues surprisingly do not replace the CI step before the automerge, but instead adds a separate later step to it. So, the end flow looks like that:

1. A pull request is created.
2. CI runs on the pull request’s unmerged code.
3. At any point before, during or after the CI run the PR is marked for merge.
4. If the CI succeeded, the pull request is put in the merge queue.
5. In the merge queue, a separate CI is run on the merged code.
6. If that succeeds, the pull request is merged (provided the rest of the queue also got merged before that).

Thus, when implementing the build queue, you need to think what checks you need to do before the PR even gets in the queue, and what checks you need to do in the queue.

The simple option is to just do a full CI run both times, but other options are possible such as:

• having smoke tests in the “pre-queue” CI and full tests in the “queue” CI
• skipping “pre-queue” altogether and only running the CI in the queue

I’ll explain how to proceed if you want either of those options in the next section.

Tutorial: how to set up a CI for Merge Queues

Firstly, you need to have a repository that is either a private repository in a GitHub Enterprise-licensed organization or a public repository in any organization. Personal repositories don’t support that feature at all, even public ones or if you have a Pro subscription.

Secondly, you’ll need some workflows. This is nice to set up before branch protection so that we can point to the existing workflow then.

We’ll need to create a pull request (pre-queue) workflow and a queue workflow.

Pull request, (e. g. put it in .github/workflows/pull-request.yml):

name: Validate pull request before putting into queue

on:
  pull_request:

jobs:
  validate-pr:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v3
    - name: Display info
      run: | 
        pwd
        tree -a -I '.git'
        git status
    - name: Run fast CI (emulated by a short sleep)
      run: sleep 5

Merge queue, (e. g. put it in .github/workflows/merge-queue.yml):

name: Validate code in the merge queue

on:
  merge_group:

jobs:
  validate-pr:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v3
    - name: Display info
      run: | 
        pwd
        tree -a -I '.git'
        git status
    - name: Run slow CI (emulated by a long sleep)
      run: sleep 300

The are two important parts here:

1. The way the workflows are distinguished are via event names. merge_group event is a new trigger that is tied to a new pull request being put to the queue.
2. The job name (validate-pr in both workflows) is important because it will be used to tie the workflows to the branch protection later. There’s no way to set different required status checks to a pre-queue and a post-queue run, so they’ll need to be named the same. This way they look like the same job to GitHub.

Finally, you need to set branch protection rules. Go to Settings → Branches and create or modify the rules for your trunk branch. You’ll need to enable “Require status checks to pass before merging” (select “validate-pr” as the required check) and “Require merge queue”.

That should be enough to get you started. Now when you create a pull request that starts the pull request CI:

You can click “Merge when ready” either before or after that is finished. This will place the pull request in the queue and run the queue CI:

Other options for CI split

If you’d like to have the same CI running both before the queue and after, you can just use one workflow, along these lines:

name: Validate code

on:
  merge_group:
  pull_request:

jobs:
  validate-pr:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v3
    - name: Display info
      run: | 
        pwd
        tree -a -I '.git'
        git status
    - name: Run slow CI (emulated by a long sleep)
      run: sleep 300

If you’d like to run nothing before the merge queue, that is not possible or I haven’t found how. So you’ll need to modify the pull-request.yml to do almost nothing, along these lines:

name: Skip validating pull request before putting into queue

on:
  pull_request:

jobs:
  validate-pr:
    runs-on: ubuntu-latest
    steps:
    - name: Skip run
      run: echo "Run skipped"

Link

There’s a sample repository I used to test the above. You can access it here: kojoru-sandbox/MergeQueues (github.com)

Good luck and happy queueing!