Design Tests & Compensation

A perspective on hiring expectations

Last week, a few folks entered into a really fantastic discussion around Take-Home Design Tests and compensation on Twitter. More specifically, whether or not those of us offering work opportunities should be paying applicants for their entries during the interview process — which turned out to be something of a split issue.

And, as in all good discussions, there was a lot of nuance surrounding the conversation. Situational dependencies that made it more difficult to do in-person interviewing, synchronization issues to overcome, as well as an opportunity to reduce bias in our candidate pool, to name a few. Which are all incredibly valid and fair points in why a Take-Home Design Test might be effective in a given situation.

Hiring is hard work. Applying is hard work. There’s almost nothing happening in that season of life for either party that’s less than borderline stressful or hopelessly manic. I mean, let’s face it: The job application process is exhausting. On both parties, really. And we can only truly know the side we’re on when it’s happening.

But through all of the nuance surrounding this topic (which was both valid and enlightening), there’s one aspect of this conversation that sticks out to me: the lack of compensation for someone’s time outside of the interview process. And that, my friends, brings us to the topic of Take-Home Design Tests and the compensation I believe should be offered therein.

Design Tests: Are they worth it?

But first, let’s talk about the validity of these exercises to determine whether or not they can or should be utilized in the first place. For the purposes of this essay, we’ll be speaking to “Take-Home” Design Tests, where an applicant is working on a given challenge using time outside of the office (as opposed to white-boarding it all live — which is its own essay, believe me). And while it won’t be exhaustive, it may be a jumping off point to those who immediately dismiss Design Tests as wholly superficial and ineffective at measuring anything other than the ego of the issuer. Speaking from experience, I’ve personally found these exercises to range anywhere from moderately helpful to damn near abusive. Which is to say that not all Design Tests are created equal; neither from implementation nor intent.

To me, the real trouble with using a Design Test to assess skill level is just how far removed from the actual day-to-day working context that experience may be; in other words, they might be a poor indication of actual performance conditions for the prospective role. While I suppose it could be useful to assess how a candidate might perform under immense pressure (do this, along with all other asks from potential employers), with next to no situational support (no current coworkers) in a topic they likely know nothing about (unless they’re lucky), I’m still left wondering just how close that is to any situational reality they might experience on the job.

And while we can’t promise that any workplace will be the perfect culmination of best practices and an enduring state of serenity, it is our job as leaders and managers to actively minimize those instances; not embrace them as if they are the working norm.

Testing someone on an extreme use case that they (hopefully) won’t experience while on the job feels like an unnecessary pressure test that seeks to be more a measure of how much time they’re willing to invest in your request in this given moment. And that is a situational equation — one that we can’t possibly hope to get the full picture of with any real clarity. Are you the only job they’re applying for? In many situations, likely not — though it may be a nice (albeit self-serving) thought. But in many scenarios, your workplace is one of a few (to many) possible workplaces that the applicant is excited to be a part of. And there’s nothing wrong with that reality.

As mentioned above, what we can generally assess from the results of a Design Test is just how much time an applicant was willing to sink into the experience; outside of all of the in-person interviewing and context-seeking. But as most of us in the working world know, time simply does not equate to value. Moreover, a lack of high investment may not mean that the applicant chose to underperform, but rather that their situational realities did not allow for the quantitative time investment that other applicants (those perhaps still gainfully employed and free from the existential pressure, for example) may have to invest in such an exercise.

That’s not to say that Design Tests are all evil or ineffective at putting some form and shape to an applicant’s abilities, but rather to assert that maybe what we’re looking to measure isn’t what’s actually being effectively measured within the process. Or to put it another way: all data gathered is not equal in its value.

What variables might a Design Test measure?

I’ve found that while Take-Home Design Tests may be helpful in very loosely determining where a candidate’s current skill set might be within the situational confines mentioned above, they may also be a poor summation of an applicant’s potential. Which, to me, feels like the real value we can cultivate and elevate in our employees.

And when I say potential, I mean the ability and desire a candidate has towards skill growth. As with many things, this is a multifaceted attribute that may be benefited by some parts innate skill, but is ultimately and effectively propelled to action by a willingness to learn and soak up knowledge. Much of that knowledge experienced while on the job or under the mentorship of an invested party.

To put that into perspective, potential is a key variable for me in the hiring process, whereas tooling proficiency ranks pretty low on the list.

This isn’t to say that a Design Test can’t be an indicator of this potential. After all, these exercises can reflect a measure of dedication and aptitude; just with a big asterisk that might also include the situational dependencies of the request. In that way, sometimes it feels like using a jackhammer to put a nail in place.

So maybe it’s worth breaking down just what a Take-Home Design Test might be helpful in assessing :

  • Time invested: I mean, it feels good to be prioritized. There’s no way around that. Put simply, we can evaluate just how much time and energy an applicant put towards our exercise by generally measuring the extent of the output. While this may make us as a potential employer feel good, we’ve already outlined some ways that time invested is an uneven variable and biasing towards time as a key indicator of success may actually remove a more fitting candidate who was not able to give quantitatively. In essence, time may be a shortsighted metric to measure against.
  • Craft versatility: Maybe you’re impressed with an applicant’s ability to work within a few areas of Design (let’s suppose visual design and illustration), but you’re unsure if they possess one or two key skills they’ll be needing at least foundational domain knowledge within (UX research, for example). A Design Test might be helpful in getting a more well-rounded view of the practices a candidate may be deploying on the job that they didn’t include in their portfolio. In this use case, if a conversation wasn’t conclusive enough, a Design Test may be a reasonable response to keep a hopeful candidate in the running that may have overlooked showcasing or fumbled with properly communicating a critical skill requisite.
  • Problem-solving: A Design Test might be useful in evaluating a candidate’s ability to work through a difficult challenge that may require more nuance and unfamiliarity than what is currently showcased. Due to the nature of a portfolio, these are often the most buttoned-up, refined pieces we’ve compiled to reflect that very best of what we’re capable of. And for good reason! However, by its very curation, some might see these as a longshot from the realities of the job. Depending on the timeline (on the spot, overnight, or even a week out), an exercise could be helpful in demonstrating an applicant’s readiness to work through difficult problems that demand nuanced solutions in a more time-boxed environment. The realities of the job might lie somewhere in-between the Design Test and the portfolio piece. The exercise, therefore, is merely another data point to measure what performance might look like along this spectrum.

This isn’t exhaustive. There are likely more insights that could be teased out of a Design Test that could ultimately be helpful to the hiring party. But the point that I’m really hoping to make here is that we, as hiring managers, need to be aware of the shortfalls of such an exercise and be more intentional about what it is we’re seeking to measure and what variables these challenges might be ineffective at appropriately assessing. And that we might pressure-test ourselves on whether or not a Design Test is actually the right next step, or if seeking context in conversation might also reveal these unknowns in a far more efficient (and respectful) way.

Bills for the skills

But here’s the real crux of the conversation here: Should we be paying those who we’ve asked to participate in a Take-Home Design Test? My answer here is a resounding yes.

And I understand the pushback in these scenarios. Paying for things is expensive! And you’re right. It is expensive. But getting design work done for you (whether it’s used or abandoned on the shelf) costs real people real time and real money. It’s the nature of business. What I would recommend in these scenarios is to limit the amount of participants at this part of the interview process. If you know that you’ve got a fixed amount to offer an individual for their time, budget just how many of these Take-Home Design Tests you’d be willing to pay for across the candidate pool.

Chances are, if you’re having most of your applicants doing one, you’re wasting a lot of people’s time — not just your own.

Use a Take-Home Design Test as a near-final step in the interview process wherein the previous context-gathering mediums fell short of the requirements for hiring. Maybe you’ve only got one headcount left and there are two candidates who, thus far, seem to be equally strong. This could be a reasonable tie-breaker. So long as you’re keenly aware of just what this test might measure well and what this test will take a poor, ineffective pulse on.

Be empathetic to the applicant’s situational realities and think about what it might look like to invest all of the time and energy you have available to not just multiple rounds of interviews with stakeholders and team members, but to also have homework to continue proving that you are who you say you are and still not get the job. Imagine what it might mean to devote hours or even days on a specific design challenge and never see a dollar back from all of that investment. Paying for the take-home time of a candidate sends a clear message that you’re intentional in who makes it to this stage and solidifies your commitment to being a good steward of that individual’s time and energy. It keeps you from blindly assigning extraneous work and ensures that every decision to assess with this medium is under some scrutiny with clearly defined intents and expectations. Be discerning! Figure out where and when these tests are appropriate and when a different medium for evaluation might be just as effective — if not more!

And it goes without saying, for the love of anything holy at all, do not use a Take-Home Design Test to actually solve problems at your company and then not pay an individual for their time or a leasing of their ideas. There’s going to be a special place in Design Hell for these offenders of human decency and I do not want to be on that bus.

I‘d recommend creating a design challenge that sits far outside of any business objectives just to play it safe. What’s more, it will also prevent you from coming to the review table with the burden of knowledge. We’ve all learned how to work within our business contexts to the extent that we might be dismissive or biased against solutions that we’ve either already tried or would prefer to stay away from. Keeping the test outside of these domains might help you be more objective and open to new and emerging ideas. And hey, it will be an actual surprise for the potential candidate which actually could assess how they show up when juggling unexpected challenges.

In conclusion

I personally don’t believe Design Tests are all evil in the hiring process. In fact, I’ve used them in the past as a hiring manager. While we did offer compensation for these exercises, I still feel that what we offered was less than what I’d consider fair and it was used far too often with candidates I had already been comfortable making the call on. This unnecessarily dragged out the hiring process, putting potential employees through yet another gauntlet they needed to nail the landing on. Upon reflection, some of those don’t feel like they were an efficient use of time or resources on either side of the equation.

My ask is that we use discernment on where these tests might be appropriate, more rigorously question the necessity during the process, and maintain some awareness of the numerous blindspots they come packaged with. May we have clear expectations on what insights a test of this nature will illuminate as well as what measurables these will not effectively demonstrate.

And ultimately, that if we do move forward with a Design Test, that we pay our prospective employees for their precious time and hard-fought energy.

Speaking of hiring, did you know that we’re hiring Product Designers at Pluralsight? Check it out!