The Coding Interview: The importance of hands-to-the-keyboard interviews


I have been interviewing candidates for software development positions for quite some time and the only real conclusion I’ve reached is this: It’s tricky business.

Interviewers must make a very important decision in a short amount of time with limited information. To be effective we must have a clear set of priorities and a plan for achieving them. Because we are hiring software engineers, and engineers write code, every interview process should be able to answer the question:

How well can this person code and how long will it take them to become an effective contributor?

As an interviewer (and an engineer), I want to be able to quantify my assessment of a candidate. I want to base it on real evidence. Finally, I want to have a sense of how long it will take this person to start contributing to our team in a tangible way.

It turns out that if you sit with a person and watch them write code for a while, you can get all of this and more in a relatively short amount of time. Based on my experience I would assert the following to the industry at large:

At least some portion of every software engineer’s interview process should include hands-to-the-keyboard coding.

In this post I will be sharing my thoughts on coding interviews and how they should be run to maximize their effectiveness.

Anatomy of a Coding Interview

Ok, so what does a coding interview actually look like you ask? Glad you asked. Assuming a 45-minute time slot for the interview, it will usually go something like this:

  • 5m Introductions
  • 35m Coding inside of a real IDE *see tips section about IDE/OS familiarity
  • 5m Wrap up

As you can see, the lion’s share of the time during this interview should be made up of watching the candidate actually code or solve coding problems.

Keep in mind that the coding interview is just one piece of the interview process puzzle. Here at HomeAway, after a candidate has been phone screened or interviewed on campus, they will typically have between 3 and 4 on-site interviews. Candidates will be interviewed by HomeAway employees from across several different teams. Each interview will have one or two areas of primary focus so that, at the end of the day, we can evaluate the candidate’s:

  • Technical/coding skill
  • Problem solving ability
  • Software design sense
  • Prior experience & technology overlap
  • Culture fit & communication skills

Introductions

Introductions are obviously important but they need to be kept to a minimum for the purposes of this interview. I will offer the following advice:

  • Always give your name, explain your role, and the team you work on.
  • Keep your own personal introduction to 1–2 minutes. Many candidates will match your pace.
  • If a candidate goes on for too long you must stop them or you will not see enough code to make a good assessment.

Coding

This is where things get real. Your job is to find out how well the person in front of you can code and the depth to which they understand the language they will be using. An effective coding interviewer will be able to affirm the following:

  • Coding fundamentals have been demonstrated
  • Problems have been solved using code
  • The candidate has been challenged
  • I have seen enough volume of code

Let’s talk about each of these.

Coding fundamentals have been demonstrated

Would you hire a senior developer that could not iterate over a list of objects and perform simple conditional logic? If you think that reviewing a candidate’s resume and asking problem-solving and design questions are going to guarantee this doesn’t happen, you are wrong. Admittedly, it is very rare but I have come across candidates for senior positions who fail to demonstrate these basic coding fundamentals.

Whatever problem(s) you ask the candidate to solve should require them to at least demonstrate basic coding fundamentals like object construction, list iteration, conditionals, inheritance, etc.

Problems have been solved using code

Fundamentals are important but at some point I want to see the candidate write code that solves a problem. This is where you are really going to get a sense for the candidate’s coding style and quality.

The problem(s) they solve need to be simple enough that they will not get overly bogged down and not write any code but at the same time complex enough that it might require a couple of iterations to get correct. My preferred approach here is to provide a couple of empty methods with documentation for what the method should do and then ask the candidate to provide an implementation.

As a convenience, I typically have pre-written unit tests (go Test Driven Development!) that invoke the candidate’s code so that we can tell right away if the code works or not. When code doesn’t work the first time, seeing how a person goes about their debugging process can tell you a lot.

The candidate has been challenged

If I finish an interview and I have not found at least one or two areas where the candidate was challenged, I consider that to be a failure on my part. Tailoring the difficulty of your coding interview can be a challenge and will often require having a pool of questions to choose from with varying degrees of difficulty.

One strategy is to ask a series of questions that get progressively more difficult. If you have a series of say 6 questions where the 5th and 6th ones require a candidate to demonstrate some pretty advanced concepts, then you will also have a shot at discovering the upper limit of their skill.

I have seen enough volume of code

At the end of an interview I ask myself, “How well can this person code and would they be able to start contributing if they started next Monday?” I find that I have more confidence in my answer when I have seen enough volume of code and I have seen the candidate use code to solve a number of different problems. More data is almost always better.

Here are some common pitfalls that sometimes result in not getting the volume of code you should expect:

  • Problems that are too vague and require a lot of design time
  • Brain-teaser problems that require considering a lot of possible solutions
  • Problems with a lot of complexity

Remember, this coding interview will not be the only interview a candidate has. Heavy problem solving and design questions are tempting but should be saved for non-coding interviews. The focus of the coding interview should be on using code to implement solutions.

Wrap Up

You should always leave time at the end of the interview to give the candidate a chance to ask questions. Time management, however, can be difficult during coding interviews. My advice here is:

  • Regularly check how much time is left.
  • Be flexible and willing to switch your questions around if something is taking too long (easier when you have prepared several questions that increase in difficulty).
  • Inform the candidate that the last portion of the interview will be devoted to questions and that you might have to halt the exercise at a certain time.

Tips for Running a Good Coding Interview

  • Have the development environment ready to go. There should be no time wasted starting applications or setting things up.
  • Close apps with alerts, like Skype. There’s nothing worse than getting a Skype alert with “How’s the candidate doing?” while said candidate is typing away on your laptop. Yes, this has happened to me.
  • Ask the candidate which IDE they use and how familiar they are with the OS you are having them use. Sometime PC users fumble with things on Macs (and vice versa) and it’s good to know this isn’t a coding skill problem, just unfamiliarity with the OS or IDE.
  • Have a project already setup with the questions you are going to ask.
  • If you are reusing a previous project, make sure you have DELETED any previous candidate answers.
  • Have a convenient way to test candidate-written code, like pre-written unit tests.
  • Get feedback from coworkers on questions before you present them to candidates.
  • Avoid questions that require excessively long explanations before coding can begin.
  • Sometimes it is nice to ask different candidates the same set of questions, especially when you are trying to make a direct comparison between them, but be cautious of overusing questions because they will eventually make their way through the interviewee grapevine (and onto Glassdoor).

Closing Thoughts

Coding interviews, when run properly, are a valuable tool for determining a candidate’s coding abilities in a very real and measurable way. Coding skill isn’t the only factor we need to consider but, for software engineers, it should be one of the most important!