Designing Better Take-Homes
How to make take-home assignments more fair, pleasant, and effective for employers & candidates
My co-founder Wayne and I started Headlight to make performance-hiring the mainstream way companies recruit talent. The best way to do that today is to use an appropriate take-home assignment as part of the screening process.
Many companies today are using take-home assignments as part of their hiring process, while others are still just getting started with this hiring practice. Here are some the best practices we’ve seen for developing and administering assignments that are fair, useful, and welcomed by candidates and hiring managers alike.
Design
1. Choose a practical task
Companies are criticized, rightly so, when they ask candidates to perform academic or theoretical tasks rather than real-world problems. For instance, instead of having developers having developers find all the palindromes in a string, a better request would be to have them to refactor an existing function or interact with a simple API.
Give them a little more context on why the task needs to be done—this helps . Ask yourself: does this assignment feel close to something one might be asked to do as part of the normal responsibilities in this role?
2. Fit the assignment to the role
I’ve encountered employers who developed a take-home assignment for full-stack web engineers and then tried to use it on iOS or Android developers. This doesn’t make a lot of sense. It’s annoying to have to develop new assignments but if you’re testing a UX designer on CSS or a content marketer on PPC strategies, it’s frustrating for both the candidate means a frustrating candidate experience and less useful data for the employer.
Just as it’s not super helpful to know that someone is a good swimmer if you’re assembling a cross-country team, a misaligned assignment wastes both the employer and the candidate’s time.
3. Don’t ask for free consulting
It’s natural to want to ask candidates to solve problems that are directly relevant to your company — you get to see how they think about your problems and maybe even get some new ideas as well!
The problem is, you have way more information about the problem and constraints than they do and this question sets them up to disappoint you. It’s not fair to ask candidates to overly invest in learning about your specific industry and company before you’ve shown a commensurate investment in them. This is doubly true when you aren’t compensating candidates as it can be seen as asking for free consulting. Consider inventing a scenario in an adjacent industry — if you’re an e-commerce company, have your take-home feature a travel deals site, etc.
Note: In later stages (e.g. after onsite interviews) or for executive roles, it can be appropriate to use an assignment that’s more focused on the business.
Workflow
1. Set and enforce a reasonable time limit
One of the biggest gripes candidates have about take-home assignments is how much time it takes to do a good job. Even if you provide guidelines to stay within a certain time frame, without enforcement, there is an arms race to invest 2x, 3x, 5x more time and submit something stronger while implying you stayed inside the suggested time frame. This arms race causes candidates to drop out or face unfair evaluation and your take-home is now just optimizing for people with free time.
While some companies swear by open-ended assignments, we believe this policy distorts results. We’ve seen employers get a great deal of signal from assignments with 2–4 hour windows. Headlight allows you to enforce a time limit but if you are not able to do so directly, you can at least delay sending candidates the assignment details until they confirm they’ll have the free time to complete and return it in a few days.
Note: if you notice a meaningful drop in quality when you start enforcing time frames, that may be a sign that past candidates were spending longer than suggested—not that your new candidates are somehow worse!
2. Consider compensating candidates.
While this is not yet a commonplace practice for take-homes, it can really help you stand out from other employers if you offer to compensate candidates. Companies are often concerned about candidates dropping out at the take-home step and if you can save a couple of promising candidates from bailing for maybe thousand bucks all together, it’s a no brainer.
You can go with a lump sum or pay a flat per hour rate—Wordpress does $25/hr for every role (but you’re welcome to bump it up). Make sure to offer the option to donate an equivalent amount to charity, this is particularly important for H-1B visa holders and candidates who have no-moonlighting policies.
3. Acknowledge receipt of the assignment
Show candidates the courtesy of acknowledging their work and letting them know when they should expect to hear back about next steps. Nothing is worse that spending a weekend tackling a take-home, only to get weeks of radio silence after handing it in—even if you finally tell them you love it and you want to bring them in, they’re not going to be feeling as good about this job.
4. Don’t ask for revisions
We’ve heard of employers requesting edits and revisions after they’ve handed something in, which you can imagine is a huge turnoff. If you’re invested enough in a candidate to want to ask for more, an onsite might be justified, or at minimum, a follow-up phone call to discuss the assignment in detail.
Evaluation
1. Have evaluation criteria developed in advance and communicate them to the candidate
When you lay out a grading rubric in advance, you indicate that you’re treating this assignment seriously and that you will review it systematically rather than haphazardly. This also makes it much easier to review / grade take-homes quickly and consistently. Ideally, these criteria are tied to a competency matrix for that role so that what you look for in candidates is aligned with what you look for in promoting employees.
Every assignment on Headlight requires a defined set of criteria that is automatically displayed to the candidate when they’re working through the assignment and is then used during evaluation stage.
2. Have multiple reviewers who are not the hiring manager evaluate each submission
In a pinch, it’s understandable that the task of reviewing the take home submission fall to the hiring manager, but if possible, we recommended having the take-home reviewed by at least two employees who either work in the role the candidate is applying for or has intimate knowledge of what the role entails.
Everyone notices something different when evaluating work, even with a clear rubric, and having multiple reviewers helps even out these differences and it makes sense for them to provide feedback on a potential peer.
3. Hide identifiable information about the candidate when reviewing
Take homes should be reviewed on the basis of the candidate’s work only, which means if its being reviewed by folks other than the hiring manager (as suggested above), care should be taken to remove information that identifies the person (actual names or user names, profile picture, location, etc).
Gender and racial bias is an extremely widespread and well-documented phenomenon, even among progressive organizations, and a powerful way to guard against it is simply not to know. This idea was put into practice in science, as documented in the Nature article “Fund ideas, not pedigree, to find fresh insight” where a promising new background-blind grant making format encouraged researchers to submit their boldest and most innovative ideas.
4. If your assignment becomes widely known, update the assignment
The challenge with take-home assignments, which is similar to that of whiteboarding exercises or other questions, is that over time, they may get posted on public sites or shared over mailing lists and private repos. It’s worth checking Glassdoor and CareerCup to see if your questions have leaked and update them every 12–18 months.
As work becomes more complex and interconnected, the need to understand not just what someone has done but what they can do is so important. Well-designed take-homes can be a positive experience for candidates and provide important insight for hiring managers. We hope these techniques help you make yours great.
If you’re interested in having better onsite interviews and reducing time spent in the screening process, check out Headlight. And please give this piece 👏 so others can find it too!
Have thoughts, questions, feedback? Find me at @jasonshen.
