Not running structured interviews? Don’t waste everyone’s time

The case for structured interviews

As time-pressed entrepreneurs, we wing it with spontaneous, off-the-cuff interviews. After all, why spend too much time preparing for the interview, when we are not the ones being evaluated — or so we think. But unstructured interviews may actually be wasting your time, losing you talent and harming your hiring practices.

Structured interviews are a better predictor of job performance than traditional unstructured interviews. Across hundreds of job interviews from multiple studies, the correlation between unstructured interviews and on-the-job performance was weakest. As interviews became more structured, the correlation improved. When you prepare to conduct an interview, you get better at predicting if the candidate can succeed in the role.

Correlations from Huffcutt and Arthur — a score of 1 means interviews are a perfect indicator of job performance, and 0 means the two are independent of each other.

By determining the topics, content and questions for an interview, we can secure better outcomes than if we freestyle it. It does not have to be overly prescriptive. The difference between asking a fixed set of questions across all interviews (most structured) versus picking from a list during the interview (slightly less structured) is minimal: having the questions planned in advance matters most.

In fact, one structured interview collects as much data as 3–4 unstructured interviews (Schmidt and Zimmerman). Talk about a productivity hack! Evaluating a candidate when you are well-prepared significantly increases your chances of finding the right fit.

Develop an interview structure

Now we have talked about the why of structured interviews, let’s talk about the how.

The first step of structuring interviews is to answer a simple question — what does good look like?

This is an iterative process where all hiring stakeholders (founders, hiring managers, reporting managers, etc.) need to come together and align. By documenting what good looks like and coming together to discuss, expectations can be set, allowing clear and purposeful interviews.

Gartner provides a useful framework on how we can define good — knowledge, skills, abilities and other characteristics (KSAOs):

• Knowledge: the body of factual or procedural information, such as knowledge of foreign languages or programming languages
• Skills: the capabilities required to perform tasks accurately, such as typing speed or driving ability
• Abilities: stable characteristics that can include cognitive, sensory and physical abilities, such as empathy.
• Other characteristics: traits that do not fit into the other categories, including values, work style, personality and degrees and certifications

To create a structured interview with KSAO, there are some considerations:

• Define what is must-have, nice-to-have, and we’ll-teach-you-on-the-job. Depending on each role, industry or company, the 4 dimensions should weigh differently. Knowledge may be a prerequisite for software engineers (if you don’t know any programming languages, you won’t be able to write software), but may be less pertinent for a B2C sales role (you can learn specifics on the job, but skills probably matter more)
• Categorize each interview question in one of the four categories of KSAO
• Evaluate each question’s relevance to the job: what does this question test in the candidate?
• Review each interview question: will the candidate’s answer help me recognize the right person for the job?
• Develop a rating scale so you can objectively rate each candidate after interviewing. Qualitative comments add color, but rating on a scale helps avoid recency effect (more on this later)

Ask situational questions

Interview questions are the butt of many jokes (and memes), and for good reason. To better collect data that determines if the candidate is a fit, I recommend situational questions.

Asking someone about their 5-year ambitions rarely yields helpful insights about the individual. This is exactly the kind of question that you can find 101 template answers to with a simple Google.

Instead, make the effort to create relevant questions such as:

• Tell me a time when you were in an ambiguous situation. What was the situation and how did you respond?
• What would you do if a customer takes your recommendation as a critique and reacts defensively?
• When was the last time you missed a deadline? How did you manage the situation?

These situational questions indicate how they respond to similar situations on the job, and are particularly effective when evaluating the Skills and Abilities elements of KSAO. Setting the right context and defining the right situations requires careful consideration, so structure the interview!

Know and avoid your blindspots

Structuring interviews offers an opportunity to identify and avoid blindspots. Several common oversights include:

• Biases: research has proven repeatedly that we make biased decisions. Unsurprisingly, female and under-represented candidates are often victims of this:: “received less credit for prestigious internships in all fields”, “less likely to accept the job offer”, and so on. We must call out those biases and set guardrails to avoid them.
• Recency effect: hiring is tiring, and our brains are wired to recall information we saw most recently. When we are interviewing our 5th or even 10th candidate, we tend to forget our first few, which is why it’s imperative to adopt rating scales and document interviews
• First date effect: first dates tend to be sunshine, rainbows and romance. But this rose-tinted glasses effect is probably because we know very little about the person. We rush to conclusions — up to a third of interviewers make hiring decisions in the first 5 minutes. Let’s take it slow.
• Image management: practically all candidates are on their best behavior during interviews (McFarland et al.; Levashina & Campion). For the eloquent or glib, it is important to recognize that candidates are presenting their best selves. Try to dig beneath the veneer and ask questions on their human side.

How should we respond to these blindspots?

Triangulation: create more sources of information when assessing candidates. Combine KSAO ratings with resumes, skills assessments, trial projects or references. To truly ‘triangulate,’ set up at least three separate interviews with different interviewers (Rogelberg).

Mean Reliability of Unstructured and Structured Interviews for Varying Number of Interviewer-Occasion Combinations from Schmidt and Zimmerman. Reliability calculated using the Spearman-Brown formula

The takeaway

Finding the right candidate can be challenging, but running structured interviews increases the likelihood of a positive outcome. In summary:

• Unstructured interviews waste time: yours and the candidates’. Structure interviews before scheduling them.
• Define what good looks like: you can’t measure without a yardstick.
• Ask situational questions: understand the candidate in context
• Know your limitations and triangulate information with other interviewers to decentralise hiring decisions.

If you have come up with a method that works well, or are keen to continue this conversation, hit me up at qinen@saisoncapital.com

#humancapital #hiring #recruiting #interviews #interviewing #startups #biases #venturecapital