Take-home Coding Assignments Are a Waste of Time

Photo by Andrea Piacquadio from Pexels

A good interview process for software engineers should reliably assesses whether candidates can write code to solve the kinds of problems they’ll face day-to-day. The question will arise: why don’t we give candidates a coding test as a take-home assignment? This would provide the candidate with a much more “natural setting” to demonstrate their skills outside the pressure of an interview. A larger and more open-ended problem will allow for a better-rounded assessment. And besides, if someone doesn’t want to do this test, isn’t that a sign they’re not truly interested?

Gonna have to “Nope” that idea! (This card is from “Exploding Kittens,” brought to you by the fine folks behind The Oatmeal)

Take-home projects sound good in theory, but in practice it’s difficult to make them a worthwhile part of your interview process. Potentially bad outcomes are:

• A promising candidate abandons the hiring process because the at-home project was onerous or annoying → any assignment must be well-contained, not too long, and engage candidate interest.
• Many candidates lack quiet time outside of work and cannot commit to homework→ such a program must be developed to minimize bias.
• We give candidates a significant project assignment, but our evaluation is always a wishy-washy “the code looks ok, but it could be better” → there need to be upfront criteria where it’s possible to score as very good or as very bad; “meh” adds no signal and is a waste of time.
• The “at-home assessment” isn’t really a project, it’s just a series of small online quizzes to test narrow language skill and filter out people→ ugh, that’s a terrible candidate experience, and it’s hard to validate that such quizzes aren’t filtering out good people or accepting mediocre ones.

For an at-home assignment to be successful:

• It must be sized correctly: it’s an interesting project that’s bigger than a quiz, but not more than about 2 hours, and it can be scheduled flexibly.
• It’s an interview, not a filter: there must be consistent scoring criteria and the ability for a candidate to dazzle; it’s not just a check-the-box gate.
• Your team trusts and pays attention to the exercise: people read the code critically, and it factors into the hire/no-hire decision.

That’s a tall order! Frankly, I haven’t seen many companies pull this off well. I’m not saying it can’t be done; but my experience is that 8 out of 10 times it’s done poorly, which does not respect the candidate’s time and creates a false security that the candidate’s coding skills have been assessed properly. It’s disappointing when a team pushes forward with at-home projects for the sake of “why not collect as many data points as possible?” but then fails to effectively use that information. Until a company has an established and mature hiring process, and there is enough capacity to approach them quite thoughtfully, my stance is that at-home assignments are a waste of time.

Focus on the in-person coding exercise

Regardless of whether you have an at-home assessment, the most important section of your interview process will be an in-person coding exercise. This lets you verify the candidate’s technical competence as well as their ability to communicate effectively and collaborate with a fellow engineer. I’d suggest that companies devote their attention towards making the in-person interview as useful as possible, and skip an at-home exercises (especially at first).

The highest-signal type of exercise I’ve seen is a 90 minute coding interview that uses a test-driven-development framework and is presented as a pair programming collaboration. The interviewer poses the problem statement and provides a bare-bones framework that includes a few initial unit tests. The candidate writes code to solve the problem and make the tests pass, then the interviewer adds more requirements and tests. This interview works well because:

• There is plenty of opportunity for the candidate to write code.
• 90 minutes in length is the sweet spot between being able to go in-depth while not exhausting everyone.
• To succeed, the candidate has to engage with the interviewer as a peer.
• The setup puts the candidate at ease; instead of facing a blank slate and an antagonistic interviewer, they start with with skeleton code and a collaborator.
• After giving this problem to a few candidates, it’s becomes pretty clear how to evaluate what bad-ok-good looks like.

Incorporate any at-home exercise into the overall interview

If you do go the route of requesting an at-home exercise, I encourage you to incorporate it into your overall interview process rather than treating it as a stand-alone stage. The most successful process I’ve seen asks a candidate to do homework to solve a problem, and then to further discuss and workshop that project as part of the on-site interview. This includes adding new wrinkles to the requirements to see how the candidate would modify their solution. This hybrid model forces both the candidate and the interviewer to engage more deeply with the project code, which honors the effort the candidate was asked to put in.