What I’ve learned interviewing 500 people: the interviewer skills ladder for high growth software companies

It’s possible that the only thing engineers hate more than a nasty bug is interviewing. Interviews are interruptions from deep work, involve a somewhat awkward set of interactions with a complete stranger, and after you’ve done a bunch, they become pretty monotonous. They’re everything most engineers were trying to escape by becoming engineers.

Yet engineers at high growth startups will easily spend 10%-20% (or more!) of their time interviewing (do the math: 2 interviews at 1 hr interview + 1 hr prep is already 4 hours). At hyper scaling companies, some engineers will get tapped to do a tremendous number of interviews because it’s the highest leverage activity they can do. For example, Albert Ni interviewed every full time engineering candidate at Dropbox for about a year between 2011 and 2012, just as Dropbox’s engineering team was hitting an inflection point in its growth.

Impactful work done by resentful people is not a recipe for success.

Sure, we can optimize the hiring funnel or otherwise reduce the need to hire — but it’s never going to go away. So let’s tackle the other side of the problem: how can we make interviewing just a little less awful?

One big problem with interviews is that most interviewers pretty quickly get to point where they don’t feel like they’re learning much doing interviews. At the start, the idea of doing interviews can be fun and exciting — it’s a new skill, and an important thing for the company. But once you hit a state of basic competence, it becomes a boring grind.

The thing is, the skill range for interviewing is much wider than that — some people are far better interviewers than average, people who single-handedly make others want to work with them. But most folks don’t really have a clear sense for what getting there might look like.

And by the way, developing as an interviewer isn’t just about work. It’s also a useful life skill — if you’re hiring an event planner, having work done on your house, or otherwise need someone who knows how to do something you don’t, you’re going to have to evaluate people.

This skills ladder for interviewing provides a way to think about how to grow as an interviewer.

There are two critical skillsets:

• Attracting talent — making the candidate want to work with you
• Spotting talent — accurately assessing whether you want the candidate to work with you

Interviews are a two-way evaluation, and every interviewer has the opportunity to reinforce — or ruin — a candidate’s positive impression of a company.

Across these skills, there is a natural progression from beginner to expert.

The Beginner

At the very earliest stage, an interviewer’s first and primary goal is to avoid a bad candidate experience, handle the social graces and customs of an interview and create the right conditions to evaluate a candidate. Have you ever left an interview confused about what the interviewer was even trying to get out of it? That’s the experience of interacting with a beginner.

Beginners can be actively harmful in the hiring process. I was once interviewed by two people, one of whom said literally nothing for the entire interview. (They were shadowing the question to learn it. An introduction would have been nice!) As I struggled with the question, I kept wondering, “is this person being quiet because I’m just doing this all entirely wrong?” The overhead of thinking about the interviewer was distracting enough to make an already-challenging problem much more difficult. It was the most stressful interview I’ve had, as well as one of the most memorable.

Beginners are learning their first interview questions and how to write up a clear summary of their assessment. As a hiring manager, when I see interview feedback focusing on candidates with sloppy handwriting or not knowing the exact syntax for a particular programming language construct when writing on a whiteboard, I take that as the sign of a beginner.

Fortunately, most beginners know they are beginners — interviews are just as awkward and difficult for them as for the candidate.

Reaching competency

As an interviewer develops competence, they move from being a pure liability to, sometimes, an asset — creating neutral to positive experiences for candidates, keeping the interview on schedule, and engagingly explaining their work at the company.

Competent interviewers conduct unremarkable interviews for most candidates. The problem with competent interviewers is two-fold. First, they’re not likely to spot unusual talent, which means you might miss out on star hires. The second is that your competitors might have more compelling interviewers that make candidates actively want to work for them.

Coming out of college, I interviewed at several of the big tech companies of the day. I remember almost none of my interviewers, either positively or negatively, because they were competent but not exceptional. Competent, but not exceptional, interviewers means the company looks competent, but not exceptional, too.

Competent interviewers will have clear write-ups and can handle most of the normal paths an interview question might take. If the candidate has an unusual approach, though, that might spell trouble for them even if the candidate is strong. As a hiring manager, when I see competent interviewers writing feedback for candidates coming from typical tech companies, I have confidence that they’ll evaluate them well. It’s when the candidate’s background is a little unusual that I start to worry that we might be missing things.

If you’re hiring for a high growth company without a strong tech brand yet, this is a problem for you. You are going to have to find strong candidates who may not have fancy schools or companies on their resume, and you’re also going to have to fight hard to make those candidates want to work for you. Competent interviewers might work at established companies, but they’ll kill you at a smaller company.

Competency is often where an experienced interviewer will end up if they don’t work to get to the next level. One reason is simply that most interviewers get very little feedback, so it’s easy to get stuck here. If interviewing is boring, and you aren’t learning, there’s a good chance you’ve gotten stuck in this bucket.

Gaining proficiency

Proficient interviewers are where you start to separate from the pack — creating compelling candidate experiences, connecting with the candidate over common ground, and building rapport through honest answers to tough questions. Proficient interviewers are also better at evaluating candidates — they’ll know when someone is just taking an off-beat track and how to probe the candidate in a way that feels constructive, rather than aggressive. When a candidate is spinning out, they’ll know how to pull them back and give them a shot at showing what they can do.

You’re more likely to remember interviewers who were proficient at attracting talent, even if the interview itself wasn’t memorable. Why? Because you’re more likely to accept a job from a company where you like and connect with the interviewers.

One way you can tell you’re proficient is that recruiters and hiring managers try to find ways to include you in their interview loops because they know you’ll both get the candidate excited and be able to assess their work fairly, even if they take an unusual approach.

As a hiring manager, I can tell a proficient interviewer by whether they’re able to really get at the “why” behind their decision in the interview — are they able to spot a constellation of one-off data points that make up a trend, or notice when the candidate really shines in an area the interview wouldn’t normally touch on?

Honestly, this is probably a fine place for the average interview to be, and it should be just a bit more interesting because you’re starting to evaluate candidates not just against a script but engaging more with them as the complex humans they are.

Becoming an expert

At Dropbox we had one interviewer candidates would consistently name as the person they met during the interview process that they really wanted to work with. It was amazing — and it worked really well for her because she’d sometimes end up having candidates ask to join her team instead of the role they were initially interviewing for. One coworker I talked to about this person said that he literally stopped interviewing at other companies after he met her. She brought the incredible energy and candidate connection of a true talent magnet.

Think of the superpower it gives your company when there’s someone who attracts others like that.

Expert interviewers are often the reason a candidate joins a company. Often, they spot a candidate that might otherwise have gotten rejected but has some hidden strengths — several times at Dropbox I witnessed an eagle-eyed talent scout push for a candidate who became an exceptional hire, and one that we’d have missed out on without that voice. They’re also talent magnets — people want to work with them because they bring tons of positive energy, make a deep connection with the candidate, or otherwise show that they’d be a fantastic coworker. They might look like naturals, but most experts have worked hard to hone their pitch and work hard to build early rapport with a candidate.

Coming out of college, a major reason I chose the completely unknown startup I went to work for was that one of the engineers was so smart, energetic and engaging in the interview process that I knew that I would learn a tremendous amount from him. (And I did!) We bonded over our mutual love of the book Programming Pearls and I was won over by his infectious energy and enthusiasm. Remember how I said competent interviewers at startups can kill you? Expert interviews can make all the difference.

When I interviewed at Dropbox, I got excited about the company even before I came onsite because my primary contact was someone who was incredibly welcoming and enthusiastic to talk about what Dropbox was up to. We ended up with multiple back-and-forths about Dropbox’s technical stack and challenges. The more you can share that really excites a candidate’s mind, the better. Of course, when I got on-site, the conversation naturally continued from there.

One key trait of an expert talent magnet is deep empathy for where a candidate is coming from. The ultimate goal when recruiting someone is to make it easy for them to imagine themselves working with you and having a great experience. Part of doing that is deeply understanding what they care about (which may not be what they say they care about) and avoiding all the little miscues that can turn a candidate off.

How does it all fit together?

While the descriptions above capture most of the key behaviors, there are a lot of specifics. The sections below lay out a more detailed breakdown and show how each level of skill compares.

If you’ve gotten bored with interviewing and feeling like it’s just a grind, make a game out of improving:

• Find a bullet point you’re not good at
• Work at improving it, until you’ve mastered it
• Find another bullet point to tackle
• Collect all the bullets within a level and then move up!

The ladder

Attracting Talent

Beginner

• Learning how to create a neutral to positive candidate experience throughout the interview while evaluating the candidate — sometimes negative reactions may leak through
• Handling the basic social graces of the interaction — physical comfort, greetings and candidate handoffs, eye contact, etc.
• Learning how to talk about the company and the interviewer’s role in an engaging way — e.g. may get into too many details

Competent

• Ability to create a neutral to positive candidate experience for most candidates
• Keeping the interview on a schedule that allows the candidate to reach a sense of closure at the end, even if they struggled with the question
• Able to convey an engaging description of the interviewer’s own role and reason for being at the company
• Able to avoid sending out negative vibes to struggling candidates

Proficient

• Ability to create a consistently positive candidate experience
• Able to connect with candidates over common interests or experiences
• Can answer more challenging questions from candidates (e.g. about the negatives of a company) without improperly setting bad expectations for the candidate
• Ability to consistently frame questions and probe the candidate on their knowledge in a non-confrontational way that feels engaged, collaborative or curious

Expert

• Talent magnet — able to get candidates excited about the role, the company, and the potential to work with the interviewer
• Often brings a great deal of positive energy to the interview
• Usually good at sussing out a candidate’s interests and connecting with them on a personal level or using that knowledge to frame the role or company
• Well aware of what does — or doesn’t — resonate with candidates and adjusting the pitch over time

Spotting Talent

Beginner

• Learning to create the conditions for an effective evaluation
• Learning some core interview questions and the rubric for evaluating those questions
• Able to set the right up-front expectations for the interview structure and expected nature of answers so that the candidate is set up to shine
• May let candidate spend too much time in the wrong areas of a question
• Able to capture high level notes and summarize the flow of the interview along with a tentative conclusion
• Likely only peripherally aware of possible unconscious biases

Competent

• Able to evaluate typical candidate responses effectively
• Solid grasp of several core interview questions and ability to consistently apply the rubric to them
• Effectively managing time by keeping the candidate from getting bogged down in the wrong places or for the wrong reasons
• Knowing when to hint, and when not to, to set the candidate up for success
• Taking thorough notes without detracting from interactions with the candidate
• Summarizing the results of the interview into a clear, evidence-based write-up that explains the rationale for a yes or no
• Awareness of own unconscious biases, and factoring into decision making

Proficient

• Able to evaluate non-standard candidate responses effectively
• Strong grasp of a number of interview questions and their rubrics, as well as the ability to consistently apply a meta-rubric of general expectations across all interviews, and use it to evaluate candidates who go take unusual approaches to a question or who demonstrate patterns of behavior, either positive or negative, that aren’t captured by a formal question write-up
• Dealing appropriately with struggling candidates or difficult candidates (e.g. who talk a lot, or go off-track)
• Starting to shadow and reverse shadow interviews that require more subjective evaluation (e.g. behavioral interviews) or have a more open ended problem space
• Detecting ambiguity and asking the right follow-up questions while also creating the appropriate amount of “space” for the candidate to show what they know
• Consistently accounting for own unconscious biases
• Keen sense of what skills the organization requires; able to explain hiring recommendations by putting the candidate’s abilities in the context of the organization or team’s needs

Expert

• Able to evaluate a large range of candidates effectively, including senior roles or roles that the interviewer herself hasn’t held
• Able to evaluate questions where there are no single clearly correct answers and subjective judgment is required on a case-by-case basis to assess whether the candidate did the right thing
• Objectively evaluating subjective interviews and understanding how to separate a candidate’s presentation/speaking skills or environment from their actual job skills and personal contribution

Finally, if you’re interested in going even deeper into the art of recruiting, you might also like The Holloway Guide to Technical Recruiting and Hiring, which (full disclosure) I contributed to.

Thanks to Alex Hoffer, Aline Lerner (of interviewing.io), Albert Ni and Pete Belknap for their feedback on this post.