“Live coding” test : A good idea ?

I was recently reading a LinkedIn post in which the author asks how developers feel about doing 20–30 minutes of live coding during their hiring process. There were a lot of pushbacks as well as strong reactions in the comments. I wasn’t surprised. I know that live coding exercises are one of those “hot” topics in the tech community, with a lot of diverging opinions. If you’re wondering whether it’s a good idea or not to include this kind of exercise in your hiring process, it’s important to understand why developers tend to avoid them. But at the same time, you’ll need to reflect on what relevant info it will provide for your candidate evaluation.

Why live coding has such a bad reputation among developers

Note : “Live coding” is a broad term, so I’ll explain what I mean by that. It’s any problem statement that you’re given the day of the interview that you need to resolve, by writing some code with the guidance of the interviewer.

This exercise has a bad reputation for a reason: developers often have had a bad experience with them.

First, interviews can be stressful at any stage. Some people easily lose control because they feel they need to find all the answers quickly.

Second, the live coding test doesn’t match closely with the developer’s day-to-day activities. This might be due to the coding format (Coding on a whiteboard), by the nature of the problem to be solved (implementing “exotic” data structure, a complex algorithm, …), or by the tools used (language/framework you don’t know, specific IDE or platform for coding exercises). If developers feel they wasted their time in a live coding interview, that would mean there was not an explicit agreement of the expected skills for the job. The requirement might be missing from the job ad or the interview format/content was badly designed.

So, should you avoid live coding exercises at all costs?

Live coding should be seen as a tool in the interview toolbox. And like all tools, it will work well in certain contexts and will be totally irrelevant in others.

So when does it make sense to consider a live coding session?

I think it makes sense if some of the following are true:

• Synchronous collaboration activities like pair programming, pairing on specs, helping a colleague who’s stuck on a problem, etc. are common activities in your current team. Some skills you’d like to check include:
  1 - Oral communication: how easy it is for the candidates to explain their thought or their understanding of a problem?
  2 - Openness to feedback or different opinions: how do they react when they are faced with other opinions or critics of their work?
• You want to evaluate how the candidate is thinking about a problem and the steps they take when implementing a solution. What are the elements they bring up first when thinking about the problem? What questions do they ask? Do they take a test-first approach when implementing a solution? How do they build a solution incrementally?
• It could be a good complementary exercise to evaluate if you’d enjoy working with this person.

How much you value each of the previous points will help you determine how useful a live coding exercise will be to your hiring process. Another thing you have to remember: the test will only cover a subset of the skills & behaviors on which you’d like to evaluate your candidates. Remember to consider the candidate as a whole and design your hiring process accordingly. Brainstorming with a few developers will help you keep the candidate’s perspective in mind.

Making a live coding experience pleasant

At Shipup, our process has always included a live coding test. When we decided to rethink some parts of our hiring process last year, we wanted to make sure that we put our candidates in the best conditions possible.

We first brainstormed how we could reduce the stress a candidate might feel. It starts with giving a lot of transparency. Before the interview, we let the candidate know what the test will be about, how it will be conducted, by whom, what tools we will use, and what skills we will test. Developers can use their own device & environment setup with their language & framework of choice to mimic the environment they are used to.

All this info is sent to them before the interview day so that we’ll have another occasion to reassure them if they have questions.

Making the logistics go smoothly is just a start. We want to show our caring attitude at each stage of the process. After all, we all want the interview to go well and remove any impediment that would bias our evaluation.

There are many moments when we can help the candidate feel safe during the live coding. First, we explain that there is no “finish line”: Our coding test is scoped in time, and it doesn’t matter in what state we finish. There is no need for the candidates to look at the time and feel pressured.

Another thing is that we’ve designed some short exercises that increase in difficulty. One of the pros is that this helps candidates build confidence quickly. We try to let candidates choose the exercises on which they want to work, and at the same time, choose ones the interviewer hasn’t seen before. We think pairing with someone who doesn’t have more knowledge of the exercise reduces the inevitable inequity of the hiring process.

When a candidate gets stuck, we provide some guidance to unblock them and adopt a tone to suggest it doesn’t really matter that they didn’t find the answer by themselves.

Additionally, we might feel that sometimes it’s not even worth completing an exercise if the candidate has already clearly explained their reasoning, written the essential part of the code, and we see there wouldn’t be any value in making it work at any cost.

Finally, we’ve also tried to find a good balance between designing exercises that resemble the daily tasks of our developers, yet don’t require any domain knowledge that would make the problem hard to understand for candidates.

Our talent acquisition team always seeks feedback from these interviews, so we can find new ways to make them even more enjoyable. If you have any ideas from a developer perspective, we would love to hear them and see how to integrate them into our process :)

Interested to start a new challenge at Shipup ? We’re hiring! Check out our job openings at careers.shipup.co