Conducting Great Interviews, Part I — Preparation
“The secret of my success is that we have gone to exceptional lengths to hire the best people in the world.” -Steve Jobs
“The key for us, number one, has always been hiring very smart people.” -Bill Gates
A great organization requires great hiring, and great hiring means great interviewing. Obviously, interviewing is core to identifying candidates who will succeed on the job. But interviewing is also about sales — it’s core to ensuring the best candidates will decide to join you!
There’s a lot of buzz about AI in hiring, but human-to-human interviewing remains firmly entrenched as what gets most of us jobs. And for good reason. When done well, good interviews are the most effective way of identifying who will be successful on the job and getting the best candidates excited about your company.
Interviewing is not an innate skill. It comes from planning and collaboration. Job interviews have been studied by psychologists for decades and some of their findings will surprise you. This is the first of three posts where we’ll break down the keys to successful interviewing:
- Part I — Preparation
- Part II — In the Moment
- Part III— References and Debriefs
Preparation
Understanding the job
Good interviewing starts with a deep understanding of the job you’re filling. Specifically, understanding what knowledge, skills, and abilities are necessary for this job. To identify KSAs (aka “competencies”), start by interviewing teammates who do the job today and those who work closely with that role (“subject matter experts”).
This extends well beyond the hiring manager. Often the hiring manager is years removed from the role or might not have ever worked that role. If the role involves close collaboration with another team, make sure to solicit that team’s input. For example, if sales reps work closely with marketing, then marketing should contribute a subject matter expert to weigh in on the KSAs to assess.
At some larger companies, establishing KSA’s constitutes a job analysis involving psychologists, a cornucopia of questionnaires, and down-to-the-minute measurements of employees’ activities. If you don’t have that time or those resources (like most of us!), we recommend getting a handful of subject matter experts together for a brainstorming session. Have the group generate 10–20 potential competencies, and use sticky notes to identify the 5–10 that are most important. (If you want more documentation, we recommend an online poll, and potentially weighting answers by experience level.)
It’s tempting to lay out a laundry list of competencies. Wouldn’t it be great to fill every role with a PhD who holds a few patents, speaks at least 3 languages, and has written a critically acclaimed novel? But the more competencies identified as “essential”, the more you risk deterring good candidates. Laundry lists of job requirements have been shown to reduce applicant diversity — according to HBR, “Men apply for a job when they meet only 60% of the qualifications, but women apply only if they meet 100% of them.”
Finally, when convening your subject matter experts, build out your sales pitch for the role. This is where a long list is desirable! Different things will resonate with different candidates, so equip your interviewers with as many consistent, positive data points as possible about the job’s challenge, impact, learning, advancement opportunities, compensation, and benefits.
Remember: a short list of requirements and a long list of selling points is ideal.
Choosing your interviewer pool
Why have an interview pool at all? You could assign interviewers on an as-needed basis, as you schedule each candidate. That flexibility could make life easier. But an ad hoc approach means an interviewer might only meet 1 candidate for the role, making it hard to calibrate feedback, and leading to poorer hiring decisions.
The fix is to decide on a pool of interviewer when you kick off the requisition. Your options should be clear; they’re the subject matter experts we’ve already discussed. Resist the temptation to only include more senior employees; as Marco Rogers points out in his excellent post on interviewing, “It’s tempting to only tap your senior people to interview candidates. Less experienced people get a different signal — namely, if they think they can learn from and be led by a candidate.”
Your pool should be large enough that it provides scheduling flexibility — not every pool member is needed for each candidate. But it should be small enough that each pool member meets most candidates, and thus can make good cross-candidates comparisons. For example, if your final-round interview day requires 6 people (so say 2 one-on-one conversations and 2 two-on-ones), you should be drawing from a pool of 10–12.
For final-round interviews, especially when interviewing for soft skills, we recommend having 2 interviewers in the room or on the call. This helps calibrate scores. However, we advise against having more than 2 interviewers — it can feel overwhelming for the candidate and you’re no longer getting a high return on interviewers’ time.
Structuring the interviews
There are two methods that have been shown to predict job success consistently without bias: work samples and structured behavioral interviewing.
For work samples, the details are specific to the job family, but a few general principles apply. First, keep it to 1–3 hours. If you can avoid a take-home portion, do so. Take-home tests are disproportionately difficult for candidates with families, and they generally lead to significant candidate dropout. However, telling the candidate what to expect in advance is a good practice. Second, choose a work sample that can be done collaboratively. At the very least, choose a work sample in which the interviewer(s) can be engaged or observing how the candidate solves it. Learning how the candidate solves a problem is more enlightening than just seeing their answer.
Behavioral interviewing focuses on past experiences by asking candidates to provide specific examples of how they have demonstrated the competencies you identified for this job. You’re looking for verifiable, concrete evidence as to how the candidate has dealt with issues in the past as way to predict how they would handle similar situations in your organization. Each question should be tied back to a competency, and each conversation should cover 2–3 competencies.
Question quality is more important than quantity.
In a 60 minute overview, I normally expect to cover only 3–4 behavioral questions — but I’ll ask 4–5 follow-ups per question. I’ll also have a few backup questions, in case the candidate doesn’t have any experience related to one of the preferred questions.
We interview a lot of software engineers at Ansaro, and we’re big fans of Jocelyn Goldfein’s behavioral interviewing process. (Yes, we use behavioral interviewing for technical roles because behavioral interviewing applies to all roles!)
- Question: About how much of {defining new features} have you done?
- Follow-up 1: In your opinion, what does it mean to {define new features} well?
- Follow-up 2: Walk me through your personal involvement in {defining new features} over the past {year}?
- Follow-up 3: Have you made any mistakes {defining new features}? Pick one, and tell me the story.
- Follow-up 4: How did you ultimately deal with the situation?
- Follow-up 5: Do you think you could have avoided the mistake?
After {defining new features}, I’ll move on to {dealing with conflict}, {learning a new technology}, etc. We also make sure to provide our interviewers with anchors or example responses that they can use to calibrate when is a good/ok/bad answer. We set the following expectations: the interviewer’s job is NOT to make a yes/no recommendation on the candidate; it is to assess the candidate on the competencies they’ve been assigned to cover. In the final debrief, when they’re able to see all the feedback, they will have the opportunity (and maybe obligation) to vote yes/no.
We’re often asked, “Competencies sound ok, but what about soft skills? What about culture fit?” Soft skills and culture should absolutely be part of your competencies! In fact, that’s where you should spend most of your time defining competencies, because without definition, soft skills and culture become opportunities or invitations for interviewers’ implicit biases to surface.
We advise teams to focus on “culture add” instead of “culture fit.” Culture fit is the degree to which the candidate is similar to your culture today; culture add reflects how this candidate can improve and expand your culture. Culture add explicitly recognizes that some level of difference in values, priorities, and styles is not only ok, but healthy for an organization.
Nail the logistics
At this point you understand the job, how you’re going to sell the candidate, and how you’re going to assess the candidate. Make sure to figure out — quite literally — where and when this is all going to happen!
Who is going to greet the candidate at the start and who will say goodbye at the end of the day? Who will escort them from one room to the next? If they are going to be having any meals, make sure to inquire about any dietary restrictions.
Also, make sure your team is aligned on response timing expectations, and that you let the candidate know roughly when they should expect to hear next from you. It’s terrible to lose a candidate because they thought you were dragging your feet, and you thought you were moving quickly. Understand your competition by making sure that someone is on point to discuss where the candidate is in their process (do they have exploding offers or other deadlines?). Finally, request 3–4 references (fewer for employees with <5 years of work experience) and written approval to conduct reference checks.
Up next: Part II — In the Moment
