Ditching HackerRank For Something More

How BitGo rethought our interview process and moved past algorithm puzzles to screen candidates.

BitGo is constantly looking for bright Engineers interested in the crypto. Until recently, we relied on HackerRank assessments to screen candidates before progressing to in person interviews.

The results were underwhelming.

HackerRank is a tool useful for finding candidates adept at solving algorithm challenges, but does little to inform a candidate’s probability of success within our team.

This got us thinking about the characteristics that correlate to success at BitGo. When looking at our highest performing contributors, they tend to share two common traits — grit and clear communication.

Grit

This might otherwise be referred to as “great debugging skills” or “persistence in the face of adversity”. For engineers, I see “grit” as a curiosity and determination to unblock oneself when solving problems in complex systems.

Communication (Persuasion)

I am not referring to persuasion in the classical sense. I’m talking about the ability to build consensus among peers; the ability to motivate non-obvious solutions and get them into the hands of customers.

This typically comes in the form of writing — not just coding. And covers everything from technical design, to the structure of commit messages, to the thoughtfulness around how they introduce changes to their teammates.

And that’s where HackerRank assessments fall short — they don’t screen for these characteristics. The tool is useful at identifying candidates who happen to be good at, or have recently studied for, algorithm problems. While those skills are important, they don’t do enough to provide a sense of impact that a particular candidate would have on our team.

Our New Approach

We wanted to screen for characteristics we knew made for successful engineers and we wanted to do so without wasting work. The two big problems with algorithm challenges as interview tools are that:

• Plagiarism: The majority of these algorithm problems and their solutions can be found on LeetCode (https://leetcode.com/) (or similar tool).
• Discarded Work: Similar to Proof of Work systems, the output of HackerRank assessments weren’t useful for anything other than determining if we wanted to move forward with a particular candidate — this work couldn’t be re-used for anything else.

We wanted to change this. We wanted to screen for grit + communication while allowing the candidate to create something of value as a side effect.

Open Source To The Rescue

BitGo has a bevy of open source projects and each needs love. We wanted to use these projects to filter in candidates that show the aptitude & attitude to understand new systems, and the grit debug issues with them. And we wanted to use a traditional open source workflow to see how candidates handled themselves in terms of introducing their changes and persuading others to accept and merge them.

In December of 2019, we changed our approach and the results have been staggering. We had the team groom a number of GitHub issues across most of our open source projects and started giving them to candidates as their “code challenge”.

The impact was immediate. The quality of candidates making their way on-site improved. The team is now able to see what it would be like to work with these candidates in a real world context (and vice versa). And both parties are able to capture the value created by each solution.

If you find yourself underwhelmed by the results of HackerRank assessments, consider using open source challenges instead.