What do the best interviewers have in common? We looked at thousands of real interviews to find out.

At interviewing.io, we’ve analyzed and written about what makes a good interview from the perspective of an interviewee. But, despite the inherent power imbalance, interviewing is a two-way street.

I wrote a while ago about how, in this market, recruiting isn’t about vetting as much as it is about selling. Not engaging candidates in an hour-long interview is a woefully missed opportunity.

Solving interview questions is a learned skill that takes practice. So, too, is the other side of the table. Being a good interviewer takes time, effort, and the willingness to engage meaningfully with the other person.

Everyone has strong opinions about what makes someone a good interviewer. So instead of waxing philosophy, we’ll answer the following questions, and more, with data:

• Does it matter how strong of an engineering brand your company has?
• Do the questions you ask actually help get candidates excited?
• How important is it to give good hints to your candidate?
• How much should you talk about yourself?
• Is it true that, at the end of the day, what you say is less important than how you make people feel?[1]

Before I delve into our findings, I’ll say a few words about interviewing.io and the data we collect.

The setup

interviewing.io is an anonymous technical interviewing platform. People can practice technical interviewing anonymously. If things go well, they can unlock real (still anonymous) interviews with companies like Lyft, Twitch, Quora, and more.

Both practice and real interviews take place within the interviewing.io ecosystem. We’re able to collect quite a bit of data and analyze it to better understand technical interviewing.

One of the most important pieces of data we collect is feedback from both parties about how they thought the interview went and what they thought of each other.

If you’re curious, you can see the feedback forms for interviewers and interviewees below.

Feedback form for interviewers

Feedback form for interviewees

In this article, we’ll analyze feedback and outcomes of thousands of real interviews to figure out what the best interviewers have in common.

Before we dive in, let’s first put the value of a good interviewer in context by looking at the impact of a company’s brand on the outcome. After all, if brand matters a lot, then being a good interviewer might not be that important.

Brand strength

So, does brand really matter for interview outcomes?

One caveat before we get into the data: every interview on the platform is user-initiated. Once you unlock our jobs portal (you have to do really well in practice interviews to do so), you decide who you talk to. Our users are predisposed to move forward because they’ve chosen the company in the first place. Many of the companies we work with have pretty strong brands, but we also work with small, up-and-coming startups.

To quantify brand strength, we used three different measures:

• Klout Score (yes, that still exists)
• Mattermark Mindshare Score
• Glassdoor (under general reviews).[2]

The impact of interview outcomes relative to brand strength was not statistically significant. We found that brand strength didn’t matter at all when it came to whether the candidate wanted to move forward or how excited the candidate was to work at the company.

This was a bit surprising, so I decided to dig deeper.

Maybe brand strength doesn’t matter overall but matters when the interviewer or the questions they asked aren’t highly rated? In other words, can brand buttress less-than-stellar interviewers?

Not so, according to our data.

Brand didn’t matter even when you corrected for interviewer quality. Here’s the breakdown of the top 10 best-rated companies on our platform:

• 5 have no brand to speak of
• 3 are mid-sized YC companies that command respect in Bay Area circles but are not universally recognizable
• 2 have anything close to household name status

So, what’s the takeaway here?

While brand likely matters for getting candidates in the door, once they’re in, no matter how well-branded you are, they’re yours to lose.

Choosing the question

If brand doesn’t matter once you’ve actually gotten a candidate in the door, then what does? Turns out, the questions you ask matter a TON.

One of the questions we ask candidates is how good the question(s) they got asked were.

Question quality was extremely significant (p < 0.002 with an effect size of 1.25) when it came to whether the candidate wanted to move forward with the company. This held both when candidates did well and when they did poorly.

Here’s what the candidates had to say about the best and worst-rated questions on the platform:

The good

I liked the fact that questions were building on top of each other so that previous work was not wasted and finding ways to improve on the given solution.

Always nice to get questions that are more than just plain algorithms.

Really good asking of a classic question, opened my mind up to edge cases and considerations that I never contemplated the couple of times I’ve been exposed to the internals of this data structure.

This was the longest interviewing.io interview I have ever done, and it is also the most enjoyable one! I really like how we started with a simple data structure and implemented algorithms on top of it. It felt like working on a simple small-scale project and was fun.

He chose an interesting and challenging interview problem that made me feel like I was learning while I was solving it. I can’t think of any improvements. He would be great to work with.

I liked the question — it takes a relatively simple algorithms problem (build and traverse a tree) and adds some depth. I also liked that the interviewer connected the problem to a real product at [Redacted] which made it feel like less like a toy problem and more like a pared-down version of a real problem.

This is my favorite question that I’ve encountered on this site. it was one of the only ones that seem like it had actual real-life applicability and was drawn from a real (or potentially real) business challenge. And it also nicely wove in challenges like complexity, efficiency, and blocking.

The bad

Question wasn’t straightforward and it required a lot of thinking/understanding since functions/data structures weren’t defined until a lot later. [Redacted] is definitely a cool company to work for, but some form of structure in interviews would have been a lot more helpful. Spent a long time figuring out what the question is even asking, and interviewer was not language-agnostic.

I was expecting a more technical/design question that showcases the ability to think about a problem. Having a domain-specific question (regex) limits the ability to show one’s problem-solving skills. I am sure with enough research one could come up with a beautiful regex expression but unless this is something one does often, I don’t think it [makes for] a very good assessment.

This is not a good general interview question. A good interview question should have more than one solution with simplified constraints.

Anatomy of a good interview question

1. Layer complexity (including asking a warmup)
2. No trivia
3. Real-world components/relevance to the company’s work are preferable to textbook algorithmic problems
4. If you ask a classic algorithmic question, bring some nuance and depth to the table. If you can teach the interviewee something interesting in the process, even better!

Asking the question

We also ask candidates how helpful their interviewer was in guiding them to the solution. Providing well-timed hints that get them out of the weeds without giving away too much is a delicate art. How much does it matter?

As it turns out, being able to do this well matters a ton.

Being good at providing hints was extremely significant (p < 0.00001 with an effect size of 2.95) when it came to whether the candidate wanted to move forward with the company. As before, we corrected for whether the interview went well.

Hint quality may be a specific instance of something bigger, though it’s hard to quantify. It turns something adversarial into a collaborative exercise that leaves both people in a better place than where they started.[3]

If you can’t do that every time, then at the very least, be present and engaged during the interview. No matter what the devil on your shoulder tells you, no good will ever come of opening Reddit in another tab.[4]

One of the most memorable, pithy conversations I ever had about interviewing was with a seasoned engineer. He spent years as a senior software architect at a huge tech company before going back to what he’d always liked, writing code. He’d conducted a lot of interviews over a career spanning several decades. After trying out several styles, what he settled on was elegant, simple, and satisfying. According to him, the purpose of any interview is to “see if we can be smart together.”

I like that so much, and it’s advice I repeat whenever anyone will listen.

Here’s what candidates thought of their interviewers in regards to helpfulness and engagement:

The good

I liked that you laid out the structure of the interview at the outset and mentioned that the first question did not have any tricks. That helped set the pace of the interview so I didn’t spend an inordinate amount of time on the first one.

The interview wasn’t easy, but it was really fun. It felt more like making a design discussion with a colleague than an interview. I think the question was designed/prepared to fill the 45 minute slot perfectly.

I’m impressed by how quickly he identified the issue (typo) in my hash computation code and how gently he led me to locating it myself with two very high-level hints (“what other tests cases would you try?” and “would your code always work if you look for the the pattern that’s just there at the beginning of the string?”). Great job!

He never corrected me, instead asked questions and for me to elaborate in areas where I was incorrect — I very much appreciate this.

The question seemed very overwhelming at first but the interviewer was good at helping to break it down into smaller problems and suggest we focus on one of those first.

The bad

[It] was a little nerve-wracking hearing you yawn while I was coding.

What I found much more difficult about this interview was the lack of back and forth as I went along, even if it was simple affirmation that “yes, that code you just wrote looks good”. There were times when it seemed like I was the only one who had talked in the past five minutes (I’m sure that’s an exaggeration). This made it feel much more like a performance than like a collaboration, and my heart was racing at the end as a result.

While the question was very straightforward, and [he] was likely looking for me to blow through it with no prompting whatsoever in order to consider moving forward in an interview process, it would have been helpful to get a discussion or even mild hinting from him when I was obviously stuck thinking about an approach… While I did get to the answer in the end, having a conversation about it would have made it feel more like a journey and learning experience. That would have also been a strong demonstration of the collaborative culture that exists while working with teams of people at a tech company, and would have sold me more vis-a-vis my excitement level.

If an interview is set to 45 minutes, the questions should fit this time frame, because people plan accordingly. I think that if you plan to have a longer interview you should notify the interviewee beforehand, so he can be ready for it.

One issue I had with the question though is what exactly he was trying to evaluate from me with the question. At points we talking about very nitty-gritty details about python linked list or array iteration, but it was unclear at any point if that was what he was judging me on. I think in the future he could outline at the beginning what exactly he was looking for with the problem in order to keep the conversation focused and ensure he is well calibrated judging candidates.

Try to be more familiar with all the possible solutions to the problem you choose to pose to the candidate. Try to work on communicating more clearly with the candidate.

Anatomy of a good interview

1. Set expectations, and control timing/pacing
2. Be engaged!
3. Familiarity with the problem and its associated rabbit holes/garden paths
4. Good balance of hints and letting candidate think
5. Turn the interview into a collaborative exercise where both people are free to be smart together

The art of storytelling… and the importance of being human

Beyond choosing and crafting good questions and being engaged, what else do top-rated interviewers have in common?

The pervasive common thread I noticed was hard to quantify. But it dovetails with being engaged and creating a collaborative experience. It’s turning a dehumanizing process into an organic experience between two capable, thinking humans.

That often means revealing something real about yourself and telling a story. It can be sharing a bit about the company and why, out of all the places you could have landed, you ended up there. Or some aspect of the company’s mission that resonated with you. Or how the projects you’ve worked on tie into your own, personal goals.

The good

I like the interview format, in particular how it was primarily a discussion about cool tech, as well as an honest description of the company… the discussion section was valuable, and may be a better gauge of fit anyway. It’s nice to see a company which places value on that

The interviewer was helpful throughout the interview. He didn’t mind any questions on their company’s internal technology decisions, or how it’s structured. I liked that the interviewer gave me a good insight of how the company functions.

Extremely kind and very generous with explaining everything they do at [redacted]. Really interested in the technical challenges they’re working on. Great!

Interesting questions but the most valuable and interesting thing were the insights he gave me about [redacted]. He sounded very passionate about engineering in general, particularly about the challenges they are facing at [redacted]. Would love to work with him.

The bad

[A] little bit of friendly banter (even if it’s just “how are you doing”?) at the very beginning of the interview would probably help a bit with keeping the candidate calm and comfortable.

I thought the interview was very impersonal, [and] I could not get a good read on the goal or mission of the company.

One of the most genuine, human things you can do is give people immediate, actionable feedback.

After each interview, we ask interviewees if they’d want to work with their interviewer. There’s a very statistically significant relationship (p < 0.00005)[5] between whether people think they did well and whether they’d want to work with the interviewer. When people think they did poorly, they may be a lot less likely to want to work with you and, by extension, your company.

How can one mitigate these losses? Give positive, actionable feedback as soon as possible! This way, people don’t go through the self-flagellation gauntlet that happens after a perceived poor performance, followed by the inevitable rationalization that they didn’t want to work there anyway.

How to be human

1. Talk about what your company does, and what about it appealed to you and made you want to join
2. Talk about what you’re currently working on and how it fits in with what you’re passionate about
3. Give immediate, positive feedback
4. And, you know, be friendly. A little bit of warmth can go a long way.

Becoming a better interviewer

Interviewing people is hard. It’s hard to come up with good questions. It’s hard to give a good interview. And it’s especially hard to be human in a never-ending parade of interviews.

But being a good interviewer is important. While your company’s brand will get people in the door, once they’ve reached the technical interview, the playing field is level. You cannot use your brand as a crutch to mask poor questions or a lack of engagement.

A good interview can elevate a potentially cold, transactional interaction into something genuine. It can be the selling point that gets great engineers to work for you, whether you’re a household name or a new startup.

Given how important it is to do interviews well, what are some things you can do to get better right away?

To come up with good, original questions, have your team document interesting problems. Every time someone solves a problem they think is interesting, no matter how small, they jot it down in a shared doc. These notes don’t have to be fleshed out, but they can be the seeds for unique interview questions that give candidates insight into the day-to-day. Turning these disjointed seeds into questions takes thought and effort, but the payoff can be huge.

Another thing you can do to get actionable feedback is to get on interviewing.io as an interviewer. No one will know who you are or which company you represent in our double-blind practice pool. This gives you unbiased feedback on your question quality, how excited people would be to work with you, how good you are at guiding people, and more. It’s also a great way to go beyond your team and try out new questions on a very engaged, high-quality user base. You’ll also get replays of your interviews so you can figure out what you need to do better next time.

[1] “People will forget what you said, people will forget what you did, but people will never forget how you made them feel.” -Maya Angelou

[2] It’s important to call out that brand and engineering brand can diverge wildly. For instance, Target has a strong brand overall but probably not the best engineering brand (sorry). Heap, on the other hand, is one of the better-respected places to work among engineers, but it doesn’t have a huge overall brand. Both the Klout and Mattermark Mindshare scores aren’t terrible for quantifying brand strength, but they’re not amazing at engineering brand strength (they’re high for Target and low for Heap). The Glassdoor score is a bit better because reviewers tend to skew engineering-heavy, but it’s still not that great of a measure. So, if anyone has a better way to quantify this stuff, let me know. If I were doing it, I’d probably look at GitHub repos of the company and its employees, who their investors are, and so on and so forth. But that’s a project that’s out of scope for this post.

[3] If you’re familiar with Dan Savage’s campsite rule for relationships, I think there should be a similar for interviewing… leave your candidates in better shape than when you found them.

[4] Let us save you the time: Trump is bad, dogs are cute, someone ate something.

[5] This time with even more significance!