Why we pay candidates for our Coding Challenges
There are a wealth of articles floating around the internet explaining why Coding Challenges for prospective engineering candidates are a terrible idea. An irrelevant technical exercise, an example of lazy hands-off recruitment, a manufactured problem that can be perfected in advance through online practise or an abuse of candidate time and expertise?
So, are Coding Challenges really a monumental waste of candidate and employer time? Or can they be a valuable part of a modern recruitment process? And if we need them…how do we get people to do them, and to do them without feeling resentment towards the company or the process!?
Coding Challenges are not perfect — no recruitment process consisting of a few hours or even a few days can guarantee an idyllic long-term marriage of perfect engineer to perfect company. Any Coding Challenge, even one meant to mimic the work of one or two days, will still give an employer an artificial and limited set of points from which to extrapolate a good prediction of “would we want to work with this person for 2–3+ years?”
But a well-designed and relevant Coding Challenge can give us vital information points about a candidate:
- How well they can follow instructions, understand the challenge goal, and ask relevant questions.
- How they respond to suggestions and criticism about their final product.
- An insight into their work, not the work they want to put forward, the kind of work they did as a passion project, but the kind of work you need them to do.
- An assessment of their technical capabilities.
- How they respond to the types of problems we and they will be tackling if they work for us.
But what is a well-designed Coding Challenge? Do we need an 8-hour marathon on-site challenge to make these kind of assessments? What kind of problems should our candidates be solving and how do we separate the top tech talent from candidates that have utilised the volume of online practice challenges to beat the test?
Coding Challenges tend to come in two forms:
Invented/non-company specific challenges
- These often have little relevance to the job the candidate would be doing
- Though this can test your candidate for basic skills, it does seem to be a waste of time for assessing candidates for more senior positions and for ascertaining if they have the skills needed to succeed in this specific role.
Company-specific challenges — A challenge that is related to the role or in some cases an actual problem the current team is facing.
- This challenge will certainly give an employer an insight into a candidate’s relevant skills, but the process raises its own dilemmas; what if a candidate’s solutions are awesome and you end up using it in your code? Does using a candidate to complete actual company work blur the lines of an ethical recruitment? Should a candidate be credited with their contribution after the fact (regardless of employment?).
If we accept that some form of Coding Challenge is a good idea, then it seems obvious that we would want it to be as close as possible to the “real work” of our organization. But real work takes real time. So how do we incentivise the best candidates and keep them in our pipeline?
Here is the problem:
- Many candidates have jobs during the week and asking someone to complete a 4–8 hour challenge on their weekend can be a big ask especially if they have family or other commitments.
- A lot of fantastic candidates simply will not do a free Coding Challenge. If they do work for free, that can often be correlated with a less experienced or capable candidate and as an employer you are likely losing the most talented applicants.
- As a potential employee, you are putting hours into the Coding Challenge because you want to work for the company. But if there is only a 1/200 chance of getting hired those are not great odds, particularly if you’re juggling multiple applications and multiple challenges for different organizations. You might give it a shot, but any effort is likely to be half-hearted.
So how do you fix all these problems at once?
You pay for a Coding Challenge!
Of course, it is prohibitively expensive to have 200 candidates doing the Coding Challenge if you are paying for it, but if you filter potential applications to just a handful of people then that’s much more realistic. By paying it also incentivises us as a company to be disciplined in our hiring process, it is easy to offer a coding challenge to someone who isn’t really a good fit, because for the company there is little downside, we may consciously or unconsciously bring forward candidates who have little chance of getting the job. If you are transparent about that process you also communicate to your potential hirees that the chance of getting hired is greater (increasing incentivisation.)
By paying for a Coding Challenge you not only acknowledge and respect the value of your candidate’s time, but you also change the incentive structure. The process becomes “Do this work, and we’ll give you x$ and potentially hire you”, which is essentially the incentive structure of normal work with an added bonus!
Capable people will do it, since for them it is just like normal contract work, with an added dopamine-inducing bonus of potential employment. You also encourage your candidates to put in the effort, since they know getting things right will give them a genuine tangible chance of being hired.
At MindsDB we only offer our Coding Challenge to a limited number of people (usually between 3–5 for a given role.) We pay anywhere from $100-$500 USD for the challenge depending on the level of skill and time required.
If you are interested in taking a look at the type of work we do or the type of challenges we set you can see an example of one of our coding challenges here: https://github.com/mindsdb/mindsdb/issues/442
If you’d like to be considered for a job with MindsDB you can find all of our job posts here: https://www.career.mindsdb.com
