Run updates across multiple GitHub repositories with our new auto-pr tool

Lidija Coha explains how her team created an open-source tool that enables engineers to perform bulk updates across multiple GitHub repositories.

This story was originally posted in our main blog. To see it follow here.

The Developer Enablement (DEN) team is always building new tools to help our engineers work more efficiently. Lidija Coha, senior engineer on the Developer Enablement (DEN) team, explains how her team created an open-source tool that enables engineers to perform bulk updates across multiple GitHub repositories.

Since the DEN team often needs to update configuration and CI/CD pipelines as code across multiple repositories, we wanted to create a tool that allows us to implement the changes and automatically pull requests from repository owners.

We took this as an opportunity to give something back to the open source community and build a tool that would serve this purpose, engineering it in a way that has no GetYourGuide specific logic or configuration. The end result is the auto-pr tool. It allows running code across multiple repositories without worrying about setting it up every time, thus focusing on the change itself. You can see all the details on GitHub. It’s been proven to be very effective for what we need.

Getting started is easy

Step 1. Initialize auto-pr project

All you need is an api-key which is a GitHub personal access token with repo and user:user:email scope, and a privileged (robot) account ssh key.

auto-pr init --api-key=<github_token> --ssh-key-file=/path/to/ssh/key/to/push/.ssh/id_rsa

This command will create the scaffolding:

ls /home/username/work/auto-pr-project

config.yaml  db.json  repos

Step 2. Configure to your needs

When communicating changes, be clear on why you’re making the change, what the change is, and when the deadline to review and merge is.

credentials:

  api_key: <github_token>

  ssh_key_file: /path/to/ssh/key/to/push/.ssh/id_rsa

pr:

  body: >

    Body of the PR that will be generated

    Can be multi-line :)

  branch: auto-pr # The branch name to use when making changes

  message: Replace default pipelines with modules # Commit message

  title: 'My awesome change' # Title of the PR

repositories: # Rules that define what repos to update

  - mode: add

    match_owner: <org/user>

update_command:

  - touch

  - my-file

Step 3. Write, test, and update script cycle

This is the most important step. Write the code that will be run against the directory of each checked out repository. You can use the auto-pr test functionality to test it without actually committing changes. Do this as many times as needed until you’re happy with it.

Step 4. Run it 🤖

This will execute the script and open pull requests. If you save the auto-pr database file you can revisit some of the pull requests and do things like close and reopen.

How we used the tool

Updating outdated configuration or format

We used auto-pr to sunset outdated pieces of configuration and unblock future work. An example of an automatically created PR where we consolidated the build version format across all our services.

Updating CD pipelines

auto-pr was an excellent way to deliver fixes and optimizations to our pipelines as code — provided by the DEN team to all engineers — who wouldn’t have the time to work on themselves.

Docker Hub URL replacement

Due to rate limiting introduced by Docker Hub, we created a mirror of all images but still needed to replace all existing references to Docker Hub images in a timely manner. This is where the idea of auto-pr was born, as it saved the mission teams a lot of time and avoided tedious, repetitive work.

Contributing

While auto-pr has served many of our use cases, it’s still in its early days of development. There may be functionally missing or gaps in the developer’s experience with using the commands.

We would be happy to accept contributions to improve this tool and provide an even better experience. Make sure to check the contribution guidelines first.

If you’re interested in joining our engineering team, check out our open roles.