Bringing some order to Pull Request reviews

Sakis Kaliakoudas
Jun 23, 2019 · 7 min read
Image for post
Image for post

In this article, we will review some of the tools and integrations used by the Android team here at to streamline the process of reviewing Pull Requests. Specifically, the () to automatically assign PR reviews to engineers, and some scripts to remove people from this process when they are not available, based on their slack profile status.


Reviewing Pull Requests is a fundamental part of every software engineer’s role. Plenty of benefits, both for the company but also for the engineer:

  • Catching bugs early on saves of time and thus money for a company.
  • By reviewing each other’s code, engineers improve their skill set and learn about new paradigms, patterns, tools, etc., but also learn more about the product they are building.
  • Diversity of opinions for solving challenging technical problems is always welcome.
  • Spotting repeating problems allows senior engineers to start thinking about raising those issues in team meetings, creating more guidelines around them, giving presentations or writing articles, or possibly writing custom static analysis rules to catch these problems before code reviews.
  • It can be a good way to engage with the rest of the team! : )

At the same time, we want the PRs to get merged as quickly as possible to avoid them from becoming stale and getting merge conflicts. Here at Babylon Health, with a team of about 25 engineers, we are averaging about 12 PRs every day, which means that we used to see messages like this one quite often:

Image for post
Image for post

I should note that we require 2 approvals for every PR to be merged, which essentially doubles the load, requiring everyone in the team to review on average about 1 PR per day. While this doesn’t sound like a lot, it means that unless everyone does their part, PRs might start piling up and becoming stale.


Enter Pull Reminders

Even though this wasn’t a big problem yet, one of the engineers in the team suggested trying out (which is part of the tool suite), a tool that provides integration between Github and , notifying you on Slack when:

  • You are assigned a PR review.
  • Your PR is approved or has changes requested.
  • Someone merges your PR.
  • Someone comments on your PR.
  • A PR comment mentions you.
  • Your PR has merge conflicts.
  • Your PR has failed CI checks.

It also provides a nice daily report on the main team channel to pinpoint stale PRs and who is blocking them:

Image for post
Image for post

Finally, it provides some handy analytics with regards to the PRs, such as:

  • Who is reviewing the most PRs
  • Who is authoring the most PRs
  • How long PRs take to merge
  • How many PRs are open, grouped by author.
  • How large PRs are on average, in lines of code.
  • How many lines of code the codebase is increasing by.

After trialling it for a few weeks we realized that the Slack notifications improved the situation with stale PRs, and the analytics were providing some useful and interesting insights, so we decided to get an annual subscription, before the tool became free, both for the Android and the iOS team who found it to be equally useful!


Enter Pull Assigner

A few months after getting Pull Reminders for the team, the Pull Panda tool suite was extended to include . Pull Assigner is a nifty tool that automatically assigns PR reviewers to every PR. All you need to do is create a Github Team with all the engineers that participate in the PR review process, and then Pull Assigner will assign a (configurable) number of reviewers on every new PR, based on the it currently supports.

Trying the tool, we found it very easy to set up, and we were delighted with the results. Before you knew it, reviewers were being assigned automatically to review PRs and notified on Slack, streamlining the process and making the Slack messages asking for reviewers decrease. Everything was great, apart from one thing:

Image for post
Image for post

The holiday conundrum

The Pull Assigner tool picks the reviewers from a Github Team. The problem is that people might not be working on a particular day, because they are on holiday, or maybe because they caught a cold. The tool would randomly assign these people to PRs, causing Slack messages like the one above. With the size of our team, this was happening often enough for it to be annoying. The only solution would be for someone to manually add or remove people from that Github Team based on their availability on a daily basis. On top of that, the maintainer of this Github Team needed to be rotated as well, as eventually they are bound to be unavailable themselves.


The solution

Discussing the issue in our weekly Android team catch-up, someone half-jokingly said:

“it would be cool if it used Slack statuses to decide who is available for PR reviews and who isn’t!”

This triggered a spark inside me and the next time I found a few spare hours I started hacking. This seemed possible as Slack has an of a user, and GitHub has an API to or people from a GitHub team. We just needed to write a script that would:

  • Run a few times a day.
  • Check all engineers availability based on their Slack status.
  • Add and remove engineers from a dedicated Pull Assigner GitHub team as appropriate

Using the Slack APIs

To call the Slack APIs you need to create a Slack app that operates on a Slack workspace. As we only needed to use this in our private Babylon Health workspace, things were rather easy to set up. The steps to create a new app is:

  • Go to the and create a new app, then type the name of the app and select the workspace that the app will operate in.
Image for post
Image for post
The first dialog you see when attempting to create a new app on Slack.
  • Add some permissions so that your app can access the Slack API.
Image for post
Image for post
Setting up a permissions scope to be able to read the profile status means simply selecting that scope from a dropdown.
  • Select the option to install the app into your workspace.
  • Done 😎

After this, you will get an OAuth access token that can be used to access the Slack APIs. A simple curl command like the following will do:

curl -s  -H "Content-Type: application/x-www-form-urlencoded" -d "token=<the_oauth_token>&user=<slack_user_id>"
Image for post
Image for post
After creating the app an OAuth token is generated that provides access to the Slack APIs

You can obtain someone’s user ID by viewing their profile and clicking on the three dots under their profile picture, and selecting Copy member ID.

Image for post
Image for post
The Slack member ID is a short String of letters and numbers that uniquely identifies a particular user.

Executing the curl command returns the following JSON:

{
"ok": true,
"profile": {
"title": "Android Developer",
"display_name": "sakis",
"status_text": "Working remotely",
"status_emoji": ":house_with_garden:",
// some more fields..
}
}

Note the status_emoji field, which we can match against to see if someone is not available. The 3 entries that we are looking for in Babylon are :palm_tree (🌴), indicating that someone is on vacation, :face_with_thermometer:(), indicating that someone is off sick, or :red_circle: (🔴), the emoji set by the when someone creates an event in their Outlook calendar but hasn’t set any Slack profile status. These fields are not affected by the text that the user adds next to their status.


Using the GitHub API

There are 2 options when accessing the GitHub API: you can use a username and a password (possibly using a dedicated account for this type of work), or you can and use that instead of a password. This GitHub account also needs to have Maintainer access rights to the Pull Assigner GitHub team, so that it can add or remove people accordingly. After that, it’s just a matter of calling a simple curl command:

curl -u \"<username>:<password/access_token>\" --request <PUT_or_DELETE> https://api.github.com/teams/<team_id>/memberships/<github_id_for_person>

Note that it is the same URL for both adding or removing people from a team. All you need to do is use a PUT for adding a member and a DELETE for removing one.

One final thing that you need to make sure is that you remove Pull Assigner from assigning PR reviews to the Maintainer account used in this script for adding or removing people from the GitHub team. To do this you need to go into the Pull Assigner configuration page, and add this account to the exclusion list as shown below:

Image for post
Image for post
With this configuration, the `babylonhealth-jenkins` account does not participate in PR reviews through Pull Assigner, even though it belongs to the Pull Assigner GitHub team.

Conclusion

We’ve been using this setup in Babylon for the last couple of months, and it is working well, without requiring any manual intervention. Engineers are automatically assigned to PR reviews, and there’s no case where these assignments are invalid, assuming that they update their Slack status when they are not available! :) Given that Pull Panda is now , I highly recommend that you give it a try our in your own GitHub projects.


Sakis Kaliakoudas is the Android team Engineering Manager at . If you would like to join him in building the future of healthcare apply .

Image for post

Babylon Engineering

At Babylon we’re on a mission to put an accessible and…

Thanks to Matthew Dolan, Joao Alves, and Sumit Anantwar

Sakis Kaliakoudas

Written by

Android Engineering Manager @ Babylon Health

Babylon Engineering

At Babylon we’re on a mission to put an accessible and affordable health service in the hands of every person on earth. The tech we use is the heart and soul of that mission. Follow our Medium blogs to learn more about how we do it.

Sakis Kaliakoudas

Written by

Android Engineering Manager @ Babylon Health

Babylon Engineering

At Babylon we’re on a mission to put an accessible and affordable health service in the hands of every person on earth. The tech we use is the heart and soul of that mission. Follow our Medium blogs to learn more about how we do it.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store