The Pursuit of the Well-Designed Take-Home Coding Assignment
Assessing developer talent is easier said than done. It often takes a good programmer to know a good programmer. Because the spectrum of experience is so broad in software engineering, evaluating a candidate’s skill set as part of the hiring process has become crucial.
The preferred method of testing differs among companies. Some companies ask algorithmic questions over the phone. Some prefer whiteboarding exercises on-site. Some will set up a remote pair-programming interview. Some will use some combination of these. And some will assign a take-home.
The Take-Home Assignment
One of the most developer-friendly forms of skill set evaluation is the take-home assignment. A take-home assignment is a work sample where candidates are asked to complete a task that is intended to showcase their practical skills and thought process. These tasks are accompanied by a due-date but are completed on the candidate’s own time.
When designed correctly, take-home assignments have a few distinct advantages:
They Provide an Accurate Indication of Technical Ability — There is no better way to assess someone’s ability to do the work than to assess their work (profound, I know). But seriously, there’s plenty of research to suggest that well-designed work assignments correlate with job performance later on.
They Test What Matters — The well-designed take-home assignment resembles real work, rather than an arbitrary test of algorithmic knowledge. If you’re using a tool like CodeSubmit, then the take-home allows candidates to demonstrate their skill using the same tools they’ll be working with on the job (Git, their code editor or IDE of choice, etc.), rather than “simulated” work environments.
They Facilitate Diversity and Inclusion — Because they take place in the comfort of a candidate’s home (or wherever they prefer to work) and not in your office, take-home assignments reduce the likelihood of intrinsic biases playing a role in preemptively screening out candidates before you know what they’re capable of. The playing field is also leveled for more introverted candidates, who may not perform their best in a live or face-to-face assessment.
If they’re so great, then why isn’t everyone using them to evaluate developer talent?
Good question! Poorly implemented take-homes have garnered criticism from disgruntled candidates in the recent past. Here’s why:
- They’re Perceived as Free Labor — A well-designed take-home assignment is real work, but not the kind that you then integrate into your product. At CodeSubmit, the intellectual property of the candidate’s solution belongs to the candidate and is used for evaluation purposes only.
- They’re Way Too Long — When a candidate receives a take-home that requires an entire weekend of work, the free-labor issue is made even worse. Can you blame candidates for being suspicious? A well-designed take-home is concise enough to be completed in one sitting. This levels the playing field for candidates who don’t have as much free time. The best developers rarely have a lot of time on their hands and don’t stay unemployed for long. In the worst case, your 8-hour assignment might be reason enough for a great candidate to pass on your job opportunity entirely.
- They Rarely Provide Feedback — When a developer doesn’t make it past the take-home assignment, it’s usually because they didn’t demonstrate the skill set that the hiring manager was looking for. Of course, there could be a multitude of reasons why the assessor didn’t believe the candidate was right for the job, but providing feedback in these types of situations is often a lose-lose for the company. Still, it doesn’t feel good to be told your performance was lackluster without any specific feedback, and a quick dismissal hurts much more when you spent 8 hours on a Saturday working on it. By making the take-homes more concise and requiring a shorter time-investment, the cold-shoulder stings a little less.
Okay, but what exactly does an effective and concise take-home actually look like?
Glad you asked! Here are some rules-of-thumb to help you craft a great take-home assignment:
Define the Evaluation Criteria
What exactly are you looking for by administering this assignment in the first place? You should know that from the beginning. Keep in mind that this assignment needs fit within a reasonable expectation of invested time and effort.
What skill set are you assessing? Are you trying to determine if a candidate can build something within a specific stack? What about bug testing? Your assessment should be designed around specific, predefined criteria. If you’re not evaluating it, then don’t include it in the assignment. I’ll say it again: anything that doesn’t explicitly address the skill set you are assessing should be eliminated from the assignment.
One rule holds true for all effective take-home assignments: they include some boilerplate. Don’t waste candidates’ time by asking them to craft boilerplate that you aren’t planning to spend your own time evaluating. Instead, add some boilerplate and empower candidates to spend their time showcasing the skills you’re looking for.
Keep it short
When you know exactly what you’re looking for in the assignment, you can eliminate the things that don’t matter. Keep the assignment concise. Limit the assignment to something that could be accomplished in one sitting. Not sure how long the assignment will take? Then have someone on your team work through it before you send it off to candidates. Knowing an assignment is too long now saves time re-working the assignment later (after you’ve already put off candidates).
Follow up in the on-site interview
Take-home assignments shouldn’t be something that’s completed once and never discussed again. Candidates put a lot of effort into these assignments with little upside beyond moving to the next stage of the interview process and (hopefully) landing the job. So when they do make it to the next round of interviews, bring their take-home with you and go through it with them. Ask them to explain why they chose the approach that they did, about tradeoffs they made, or what they would’ve done with more time.
Take-homes are great because they provide a lot of insight to potential employers without the performance pressure of face-to-face coding challenges. They also save the employer time. So consider using a take-home assignment they next time you’re evaluating developer talent!
And if you’re a hiring manager or recruiter yourself, check out CodeSubmit.io, where we’re providing a better way to manage take-home coding assignments.
