How to Make Hiring Decisions that Make Objective Sense

My friend Jon tripled his product team’s size in 18 months, with very little impact on culture. Here’s how.

I’m lucky to be friends with genius product guy Jon Wolf. We caught up over brunch last month, and the conversation turned towards hiring.

My team at Vungle was having a lot of trouble coming to consensus on candidates, and I was floored when Jon told me that he had solved that problem a long time ago. After he described his intuitive and elegant solution, I asked to interview him so we could share his method as best practices.

Q: You came up with a unique way of making hiring decisions. Tell me about it?

A: We came up with a system, modified from one of my colleagues’ prior experiences, where each interviewer graded the candidate with one of the following scores: -2.5, -1.5, -0.5, +0.5, +1.5, or +2.5.

• +/- 0.5 was a weak endorsement or vote against
• +/- 1.5 was a strong endorsement or vote against
• and +/- 2.5 meant you’d quit if we didn’t/did hire this candidate.

The scoring sounds complicated but we did this to remove zero as a option; you HAD to be positive or negative on the candidate. We set the arbitrary threshold that a candidate had to have an average score of at least +1.0 to move forward. We chose this threshold since an average of +0.5 meant we all weakly endorse this person, and that just felt too low for the talent we were trying to add to the team.

Q: Why did you feel this system was necessary?

A: I think one of the biggest hiring issues is subjectivity over what a “passing grade” looks like. Think about the most common ways to judge candidates: letter grades, star rating, “yay” or “nay”. With letter grades and star ratings, what’s a passing grade? Is C+ good enough? Is A- too strict? And we’re already biased as to what a “D” or “4 stars” mean. With yay/nay, it’s too simplistic and severe.

We wanted a scale that was repeatable and consistent, removing as much bias as possible. We needed something in which scores were clearly defined and easily aggregated. And the scores are all inherently personal. You’re not judging the person in absolute terms; you’re defining your own endorsement of that person. I think that this gave people the freedom to be negative, e.g. “I’m weakly against hiring this person”, without feeling like a jerk.

Q: What were the best parts about it?

A: We managed to solve multiple problems in one system:

1. You couldn’t be neutral.
2. Each score was clearly defined in real terms and inherently positive or negative.
3. It was easy to aggregate the scores into a composite score, and we knew what a passing grade looked like.

Q: Were there any downsides?

A: I can think of 3 things.

First, one rarely, if ever, says that they would quit if a given candidate was or wasn’t hired, so the +/-2.5 score was only used a handful of times. Maybe not such a huge downside, but it meant we really only had 4 possible scores for a candidate.

Another possible downside was that our +1.0 threshold meant that one interviewer could effectively sink a candidate single-handedly by voting a -1.5. That said, in a small team or company, a single strongly negative vote might actually be a good reason to turn down a candidate.

Finally, and maybe most importantly, because it’s meant to be as objective as possible, our system can seem harsh and negative…there’s no pussy-footing around negative scores here, no “C-”. Our group was a team of technical product people and it worked for us, but I could see this scale making people uncomfortable.

Q: What were the results?

A: We grew our team from 3 to around 10 in about 18 months. It was a period of quick growth, and while we might have passed on some good candidates, the team we built was both of high quality and had low turnover (voluntary or not) over the next several years, something really important when longevity matters in the role.

We avoided many of the fights that teams get into related to hiring new people, and I’ve been able to use this system or something similar applied to other roles I’ve hired.

Bonus Q: I know you have a perspective on hiring “athletes.” Can you talk about it?

A: Ah, “athletes.” Every company seems to have a huge infatuation with hiring multi-talented people who can do whatever is needed.

Like everyone else, I’d rather hire people with versatility and a broader skill set, but I think companies look for athletes to make up for their own lack of clarity on what they’re hiring for. You yourself are an athlete: you’re a great writer, an analytical thinker and problem solver, with both technical and non-technical education, but you’re a marketing leader first and foremost.

I’d hire you as my head of marketing, knowing that you can do a host of other things if duty calls, but that’s very different than simply hiring you, assuming we’ll find a good role for you once you get here. If we did that, there’s a great chance that we’re going to under-utilize you, meaning that we’re not getting a great ROI on you and that you’re probably bored and unhappy.

“Athleticism” should be a bonus attribute for a candidate interviewing for a given role, not the primary reason to hire someone in the first place.

Q: Tell us a little about your background.

A: I’ve spent the last 12 years in software at several high-growth companies, at sizes from 30 employees up to 800, both in the US and Europe.

I’ve got a technical education, but have worked all over the organization: implementation and development, business operations, evangelism and public speaking, sales and account management, but my passion is in Product Management.

I like being the product guy that understands how both the developers and the salespeople will think about this product, and I think it results in a better product and a more aligned team.

For hiring and interviewing, I’ve probably interviewed in excess of 300 candidates in my career for dozens of roles, and I spend a lot of time and effort making this process more effective for the company and more pleasant for the candidate.

If you liked this article, please click Recommend, or send me comments and feedback @asharfin. Want to work at Vungle? We’re hiring!