Hiring for new positions in a team often means looking at dozens of résumés and CVs a day. Usually this would be fine. The problem surfaces in smaller companies when hiring simultaneously for several roles while managing multiple sprints, deadlines, and tight client meeting schedules. There are simply too few people to dedicate enough time to coherently filter through job applicants to sort the best of the pile.
We found ourselves in that position. So where to begin? You simply don’t have time to look at all the profiles and résumés. Even then, a résumé is just a bunch of words. On a good day, these words might mean the same thing to an interviewer as to the candidate. On a bad day, you’re reading pulp fiction.
To be smart about our time, we issued a fun developer challenge to have any potential applicants self-screen before thinking about scheduling any interviews. Our challenge is meant purely as a quick exercise to be carried out on neutral ground and does not displace paid work.
What do we care about when hiring for good teammates?
It’s easy to come up with a few items, and these were some that are important to us:
- The person is smart
- The person is thoughtful
- The person is driven
- The person can communicate effectively
How does a developer challenge give us confidence in a candidate?
They’re smart
A good developer challenge figures out exactly how smart the candidate is. OK, not really, but they should be able to solve the challenge provided it’s simple, well defined, and doesn’t take much time to code.
When a challenge meets these standards, a candidate can finish the challenge. A better candidate can finish the challenge and has time to write her thoughts about it. An ideal candidate may finish even earlier, possibly with room left over for creative improvements (such as refactoring the solution for elegance or adding tests).
They’re thoughtful
A good developer challenge reveals how thoughtful a candidate is, relative to other candidates, in the context of a controlled problem.
Aside from assessing pure technical knowledge, a good challenge also includes hidden information in the form of edge cases, which are easily surveyed when a candidate thinks beyond the mechanical act of coding a solution.
Here, we could have several outcomes: a good candidate may submit a solution that may pass the majority of the challenge.
A better candidate submits a solution that always passes the challenge when the solution is executed.
An ideal candidate submits a solution that is proven beyond a doubt to pass the challenge, and also asks additional questions about edge cases she may not have encountered during her study of the problem.
They’re driven
Simply attempting the challenge is an indication that the submitter is willing and motivated to do their share in working with you.
That said, be sure to include the challenge as an optional task in a job description with the explicit result that the interview will be helped along more quickly if a reply to the job posting includes a solution to the challenge.
Submitting a failing solution should not necessarily be disqualifying. On the contrary, it’s a chance to reveal the applicant’s tenacity, especially if they are willing to revise the submission with your guided input.
They communicate effectively
A great submission to your challenge should be well documented: in the form of comments, annotations, or simply, a few points about how the solution works to solve the challenge.
The key is to look at what the candidate has provided to help the reviewer understand their solution. This level of expression is fundamental to teams.
It is often the harder part of the challenge. Good speaking’s hard.
Our Developer Challenge
If you’re reading this and want to put yourself up for our challenge, here’s what you’re up against!
- Your solution is a program that answers questions from our server autonomously (without human assistance)
- Each session is a uniquely generated series of 5 short questions
- Answer a question correctly to receive the password for the next question
- First 4 questions have straightforward answers
- The final question adds a constraint, but is provably solvable
- It includes a fun “cereal box prize” at the end
What steps a solution program might take
Things to look for
Hiring for middle or senior level developers, we should expect that the candidates are able to provide a solution that can pass the challenge fully at least once if the solution is run 10 times.
- Did the candidate provide a fully passing solution?
- If not, did they acknowledge where the solution may fail?
- Did they try to address or resolve the cases where it fails?
Did they identify the nature of the final question? - How was their code laid out?
Do the function names make sense?
Are they consistent? - Is it well annotated?
- How easy was it to execute and verify their solution as a reviewer?
- Did they add any extra polish into their submission?
It is always fascinating seeing the results of posing a challenge as possibility to ask more in-depth questions of the candidate. A good developer challenge is not a panacea, but simply one of the technical bits of a comprehensive process that gives a team the confidence it needs in hiring its next great member and saves a load of time in the process.
You can find our developer challenge at dev-challenge.thisplace.com
