Optimistic Interview Evaluation
This article goes deep on my personal thoughts on interview/hiring philosophy. You’ve been warned.
The other day I was conducting a practice technical interview for someone looking to land their first software engineering role. The candidate was right on the border of passing. I gave them my feedback, saying that depending on the interviewer and company it could go either way, but they were on the right track. Paraphrasing their response, the candidate asked “If they’re on the fence do you think I would get hired?”
I gave as practical of an answer as I could based on my experience.
Hiring based on “neutral” or “ok” interview feedback is going to depend greatly on the company and hiring manager. In my experience, “ok”s usually don’t lead to offers.
I’ve thought about this a lot since. I think it’s a problem that a candidate can get all the way through an interview process for a panel to arrive at a “meh.” I’ll be elaborating on my thoughts that lead me to this mindset here.
Getting to Yes or No
If you’ve done a number of interviews for any position, you’ve probably heard or asked some variant of “Should an ‘ok’ be a pass?” The response has probably been iffy at best. Unless the company has a policy on what to do with these “defaulted” candidates, the interview panel finds itself trying to figure out a path to yes or no.
The unfortunate result of this confusion is usually fairly heated arguments between the yeses and the nos. People find themselves falling into the camps of gatekeepers and advocates or any other inflammatory hero/villain labels they can think of. This is natural — Our brains are wired to defend our position when there is an opposing force.
Like many “us vs them” situations, the yes/no argument can go bad fast. I’ve been in hiring panels where I’ve had my ability to evaluate questions was directly attacked. I’m ashamed to say that even if I haven’t said it out loud, I’ve thought the same about others. If the goal of hiring is to grow your team, internal strife is certainly not an ideal side effect.
So what can we do?
Ideally, everyone should be open to each other’s views and open to criticism of their own. Untraining our own Soldier Mindsets¹ is difficult. As someone who typically strongly holds strong opinions, I can attest to this. Getting every interviewer out there to do so is probably impossible.
We’ll need a system that lets us confidently determine whether or not someone will be hired without potentially putting our interviewers directly in conflict with each other.
Mandate!
One solution is for the company or hiring manager to mandate a direction for neutral candidates. This reduces the likelihood that interviewers will wind up at each other’s throats at the cost of the panel’s freedom. Setting a default decision can be beneficial, but should your default be “yes” or “no?”
For large companies with a steady stream of candidates, it may make sense to have a default decision of “no.” Theoretically this would cut down on the number of bad hires and the opportunity cost of falsely failed candidates is absorbed by the sheer number of candidates interviewing.
On the flip side, a smaller company that desperately needs people to build things and launch a product may want a default decision of “yes.” This company may not be able to afford missing out on talent. Since the candidate wasn’t an outright “no,” it makes sense to bring them in even if they’re not the perfect fit.
I’m not convinced that mandates alone are the right way to go. To be completely honest, part of this is purely personal. That said, the majority of my rationale against just setting a mandate and calling it a day is that our unclear outcome is a symptom of a deeper problem — interviews with unclear outcomes.
Strive for Clarity
We’re never going to have a perfectly deterministic interview, but we should be able to put things in place to make our interviewers more comfortable in their evaluation of a candidate. Assuming that we actually know what we want in a candidate for a given role, interviewers must be provided with a way to confidently say “yes this candidate checks this box” for each area being evaluated.
Since we have limited time, and getting to a confident yes or no for each checkbox will be difficult if we have too many skills to evaluate. We can address this by figuring out what skills are critical for the role and which are nice to have. If we prioritize our questions to figure out these core competencies and treat the nice to haves as bonus points, we can focus our efforts on getting as clear of an evaluation as possible.
Even with the clearest questions and criteria that we can get, we’ll never be able to make a perfectly objective interview. There will always be some ambiguity. The key is to limit the scope of the ambiguity to individual skills rather than the overall “should we hire” question.
So we have one more question — what do we do when we can’t get a clear estimation of a required skill?
Optimism
I think the answer is to be reasonably optimistic. If the candidate clearly demonstrates most of the core competencies for a role, but there’s not a clear yes or no for one, I don’t think that should be held against them. More generally, “absence of evidence isn’t the evidence of absence.”
I want to reframe the way I think about interview evaluation based on what’s outlined here. Because everything needs a name, I’ve labelled this frame Optimistic Evaluation.
[1] I first heard of the “Soldier Mindset” in Julia Galef’s book The Scout Mindset: Why Some People See Things Clearly and Others Don’t. Strong recommend if you’re interested in reading about how we think.
