Bringing some order to Pull Request reviews
In this article, we will review some of the tools and integrations used by the Android team here at Babylon Health to streamline the process of reviewing Pull Requests. Specifically, the (now free!) Pull Panda tool suite to automatically assign PR reviews to engineers, and some scripts to remove people from this process when they are not available, based on their slack profile status.
Reviewing Pull Requests is a fundamental part of every software engineer’s role. Plenty of benefits, both for the company but also for the engineer:
- Catching bugs early on saves a lot of time and thus money for a company.
- By reviewing each other’s code, engineers improve their skill set and learn about new paradigms, patterns, tools, etc., but also learn more about the product they are building.
- Diversity of opinions for solving challenging technical problems is always welcome.
- Spotting repeating problems allows senior engineers to start thinking about raising those issues in team meetings, creating more guidelines around them, giving presentations or writing articles, or possibly writing custom static analysis rules to catch these problems before code reviews.
- It can be a good way to engage with the rest of the team! : )
At the same time, we want the PRs to get merged as quickly as possible to avoid them from becoming stale and getting merge conflicts. Here at Babylon Health, with a team of about 25 engineers, we are averaging about 12 PRs every day, which means that we used to see messages like this one quite often:
I should note that we require 2 approvals for every PR to be merged, which essentially doubles the load, requiring everyone in the team to review on average about 1 PR per day. While this doesn’t sound like a lot, it means that unless everyone does their part, PRs might start piling up and becoming stale.
Enter Pull Reminders
Even though this wasn’t a big problem yet, one of the engineers in the team suggested trying out Pull Reminders (which is part of the free Pull Panda tool suite), a tool that provides integration between Github and Slack, notifying you on Slack when:
- You are assigned a PR review.
- Your PR is approved or has changes requested.
- Someone merges your PR.
- Someone comments on your PR.
- A PR comment mentions you.
- Your PR has merge conflicts.
- Your PR has failed CI checks.
It also provides a nice daily report on the main team channel to pinpoint stale PRs and who is blocking them:
Finally, it provides some handy analytics with regards to the PRs, such as:
- Who is reviewing the most PRs
- Who is authoring the most PRs
- How long PRs take to merge
- How many PRs are open, grouped by author.
- How large PRs are on average, in lines of code.
- How many lines of code the codebase is increasing by.
After trialling it for a few weeks we realized that the Slack notifications improved the situation with stale PRs, and the analytics were providing some useful and interesting insights, so we decided to get an annual subscription, before the tool became free, both for the Android and the iOS team who found it to be equally useful!
Enter Pull Assigner
A few months after getting Pull Reminders for the team, the Pull Panda tool suite was extended to include Pull Assigner. Pull Assigner is a nifty tool that automatically assigns PR reviewers to every PR. All you need to do is create a Github Team with all the engineers that participate in the PR review process, and then Pull Assigner will assign a (configurable) number of reviewers on every new PR, based on the 2 algorithms it currently supports.
Trying the tool, we found it very easy to set up, and we were delighted with the results. Before you knew it, reviewers were being assigned automatically to review PRs and notified on Slack, streamlining the process and making the Slack messages asking for reviewers decrease. Everything was great, apart from one thing:
The holiday conundrum
The Pull Assigner tool picks the reviewers from a Github Team. The problem is that people might not be working on a particular day, because they are on holiday, or maybe because they caught a cold. The tool would randomly assign these people to PRs, causing Slack messages like the one above. With the size of our team, this was happening often enough for it to be annoying. The only solution would be for someone to manually add or remove people from that Github Team based on their availability on a daily basis. On top of that, the maintainer of this Github Team needed to be rotated as well, as eventually they are bound to be unavailable themselves.
The solution
Discussing the issue in our weekly Android team catch-up, someone half-jokingly said:
“it would be cool if it used Slack statuses to decide who is available for PR reviews and who isn’t!”
This triggered a spark inside me and the next time I found a few spare hours I started hacking. This seemed possible as Slack has an API to get the profile status of a user, and GitHub has an API to add or remove people from a GitHub team. We just needed to write a script that would:
- Run a few times a day.
- Check all engineers availability based on their Slack status.
- Add and remove engineers from a dedicated Pull Assigner GitHub team as appropriate
Using the Slack APIs
To call the Slack APIs you need to create a Slack app that operates on a Slack workspace. As we only needed to use this in our private Babylon Health workspace, things were rather easy to set up. The steps to create a new app is:
- Go to the Slack app creation page and create a new app, then type the name of the app and select the workspace that the app will operate in.
- Add some permissions so that your app can access the Slack API.
- Select the option to install the app into your workspace.
- Done 😎
After this, you will get an OAuth access token that can be used to access the Slack APIs. A simple curl command like the following will do:
curl -s https://slack.com/api/users.profile.get -H "Content-Type: application/x-www-form-urlencoded" -d "token=<the_oauth_token>&user=<slack_user_id>"
You can obtain someone’s user ID by viewing their profile and clicking on the three dots under their profile picture, and selecting Copy member ID.
Executing the curl command returns the following JSON:
{
"ok": true,
"profile": {
"title": "Android Developer",
"display_name": "sakis",
"status_text": "Working remotely",
"status_emoji": ":house_with_garden:",
// some more fields..
}
}Note the status_emoji field, which we can match against to see if someone is not available. The 3 entries that we are looking for in Babylon are :palm_tree (🌴), indicating that someone is on vacation, :face_with_thermometer:(🤒), indicating that someone is off sick, or :red_circle: (🔴), the emoji set by the Outlook-Slack integration when someone creates anOut Of Office event in their Outlook calendar but hasn’t set any Slack profile status. These fields are not affected by the text that the user adds next to their status.
Using the GitHub API
There are 2 options when accessing the GitHub API: you can use a username and a password (possibly using a dedicated account for this type of work), or you can generate an access token and use that instead of a password. This GitHub account also needs to have Maintainer access rights to the Pull Assigner GitHub team, so that it can add or remove people accordingly. After that, it’s just a matter of calling a simple curl command:
curl -u \"<username>:<password/access_token>\" --request <PUT_or_DELETE> https://api.github.com/teams/<team_id>/memberships/<github_id_for_person>Note that it is the same URL for both adding or removing people from a team. All you need to do is use a PUT for adding a member and a DELETE for removing one.
One final thing that you need to make sure is that you remove Pull Assigner from assigning PR reviews to the Maintainer account used in this script for adding or removing people from the GitHub team. To do this you need to go into the Pull Assigner configuration page, and add this account to the exclusion list as shown below:
Conclusion
We’ve been using this setup in Babylon for the last couple of months, and it is working well, without requiring any manual intervention. Engineers are automatically assigned to PR reviews, and there’s no case where these assignments are invalid, assuming that they update their Slack status when they are not available! :) Given that Pull Panda is now completely free, I highly recommend that you give it a try our in your own GitHub projects.
Sakis Kaliakoudas is the Android team Engineering Manager at Babylon Health. If you would like to join him in building the future of healthcare apply here.
