How and Why We Do Panel Interviews

“Besides getting several paper cuts in the same day or receiving the news that someone in your family has betrayed you to your enemies, one of the most unpleasant experiences in life is a job interview.” — Lemony Snicket

As conventional wisdom goes, job interviews are best done one-on-one. Panel interviews are awkward, intimidating, and often yield poor results as N different interviewers attempt to get N different perspectives addressed.

For a while on the Treehouse Engineering team, we made sure that all interviews were conducted one-on-one based on the above. But we noticed that one-on-one interviews aren’t exempt from their fair share of problems, especially for a distributed team:

He Said, She Said

It was not unusual for two people to interview the same candidate and come away with not only differing but opposing opinions. One member of the hiring team would express concern about the candidate’s communication style or work ethic, where another member of the hiring team had arrived at the opposite conclusion and argue that there was nothing to worry about. It’s difficult to resolve these situations since no two members of the hiring team had the same experience, and it’s often the loudest or most persistent member of the team who gets their viewpoint across in spite of everyone’s best intentions.

Scheduling Nightmare

At a more “traditional” company you can probably get people to take the day off and come into the office for a few hours of serial interviews. At a company like Treehouse, that’s not really an option — we rarely have a hiring team that’s all in one location anyway, and we’re often hiring remotely in addition to that. Trying to get a candidate in Missouri to schedule rapid-fire interviews with employees in New York and Oregon and Colorado ends up being pretty difficult, and the interview process can stretch out interminably. This effect is especially magnified in situations where there are multiple stages to the interview process.

Repetitive Repetition

While every member of a hiring team is going to have their own methods and approaches to questions, we all at heart want to answer the same “big” questions — will this person be a good fit for our company, will they be a good fit for our team, and will they be a good fit for their role? When you have four separate interviews that all at heart attempt to answer those same three questions, you’re going to get asked a lot of very similar questions that are all variations on the same theme. The candidate may get asked what is essentially the same question several days apart, and respond very differently each time (as they’ve had the opportunity to think through things). This means it’s not only repetitive for the job candidate, but adds to the “he said, she said” confusion when you try to collect results.

So a couple of years ago we decided we’d switch our formats around and focus on having one panel interview with the whole hiring team. Knowing that we could be swapping one set of problems for another, we put a lot of thought into how we’d perform these interviews:

Take Turns

This is a pretty obvious suggestion, but set a “question order” before each interview. Team member A should ask first, then team member B, then team member C, then start over again at the beginning. Not only does this make sure everyone gets equal opportunity to ask questions and that there aren’t any awkward “who’s turn is it now?” pauses in the conversation, it also gives each member of the hiring team at least two questions worth of time to collect their thoughts, listen, and prepare for the next question. This preparation time is essential to conducting a panel interview well.

Send Signals

Most of our interviews are conducted online, so it’s really easy for us to stay coordinated — we’re in a slack channel during the interview, so if I want to ask a follow-up question (even though it’s not technically my turn), I can just post that in the slack channel and no one is surprised/trying to also talk when I ask my question. If you’re conducting an interview in person, however, that option isn’t open to you (without being obviously rude), so you have to come up with a more clever approach. What’s worked for us in those situations is to lean on a cue, or signal, that you’re done with your line of questioning and ready for the next person to step in. The most obvious cue would be to thank the candidate for their answer, but the important thing is that the hiring team knows what the cue is and will be looking for it.

Organized Questions

I alluded earlier to “big questions” that we want to answer in an interview. Obviously you can’t ask those questions directly — who wouldn’t answer “Are you a fit for this position” with “well, yeah, that’s why I applied?” — but it’s still a great idea to identify those “big questions” up front so that you can organize the interview around it. We create an outline of questions we like to start with, where each of these “big questions” is a section header, e.g.:

  • Does the candidate care about getting better?
    When’s the last time you sat down to learn something, and how’d you go about it? 
    - Tell us about a time that you had an idea or proposal rejected by your team, and how you handled that.
    - etc.

This doc serves as our “north star” throughout the panel interview. Do we still have some “big questions” we need to answer? If so, the next person to ask a question should start there.

Build Comfort

When I run a panel interview, I always start by explaining exactly how the whole interview is going to work. I let the candidate know what will and won’t be expected of them, maybe crack a couple jokes, and ask them to take some time and tell us their career story in their own words. We also make sure the candidate has time at the end of the interview to ask questions of us. These are all measures that are intended to build comfort in the candidate and make the panel interview experience seem less intimidating and more conversational. Clear expectations, letting them start things off on their own narrative, and turning the table around so that they get to “drill” us all make the typical job candidate feel more at ease and mitigate some of the harshest impacts that a panel interview usually has.

Those are some of the lessons we’ve learned and applied in our hiring process, and the results have been pretty compelling for us — our interviews usually feel pretty comfortable, it’s easy to get team members who aren’t very experienced with hiring to participate in that more structured environment, and the process overall goes much more quickly and objectively than it did before. Is there anything you do or have seen that you think we could benefit from? I’d love to hear it.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.