How to hire developers: version 1 of x
Disclaimer: I’ve never before been so confident when writing something that I would want to come back at a later date and revise it. I’m so certain in fact, that I was almost tempted to put the ‘edit’ line in now to save myself a bit of time later.
Picture the scene:
I’m sat at my desk, a just-out-of-junior developer. My manager calls over to me.
“We’ve got an interview today for a new developer, I need you to come up with some technical questions we can ask him.”
I anxiously jot down a list in an excel spreadsheet, and an impulsive scoring system based on some hypothetical answers I think they might give. And what’s the single most important thing you need to know about someone before you hire them?
Can you list some of the negative points of using SASS?
Actually on reflection I don’t think that was a bad first effort. Most people brush up on the showpiece features of particular technologies. You have to work with them day-in day-out to see the cons. It wouldn’t be top of my list now though.
Anyway, that was the extensive training that I got before I first interviewed someone, and I don’t think I’m alone.
Mission critical
Despite being primarily a developer, I see hiring as by far the most important part of my job.
The majority of effort in any team is always going to be provided by the sum of the team members, over the individual managing the hiring. There is an obvious disclaimer here about quantity vs. quality, or effort vs. end-product, but it’s an unavoidable truth that if you hire carelessly every time, the output is going to plummet.
Quantity vs. quality: It’s hard for example, to imagine that Tesla would be where they are today without Elon Musk, yet he presumably supplies relatively few hours per week compared to the rest of the business.
Another way to look at it is this: Even if you assume as little as 20% variation between a good, and a bad hire, that’s the difference between a 4 and a 5 day week. Once you hire 5 people at the lower end of that spectrum, you’re operating missing 1 whole developer. I’d argue from experience that the variation can be far bigger than that, and if you end up with someone particularly deconstructive they can end up costing time from other members of the development team too. Go to the extremes and you find some evidence of the concept of ‘net-negative’ programmers, who cause more issues than they solve, and therefore actually improve the team just by leaving. On the other end you get the idea of the ‘10×’ programmer, who contributes 10 times the average. Read the comments here for a good discussion about if this is myth or reality. I have no doubt that there are some exceptional coders out there, I’m just not sure what the ‘10×’ really means, especially if the scale starts in the negative.
Given all that, I‘m surprised how many of the interviews I have been in fall in to one of the following two categories:
- The HR Interview—essentially you’re just sat in a room awkwardly filling in an HR questionnaire with a stranger. I find these are also the ones where you’re likely to get a ‘fun’ question like “If you were a biscuit, what kind of biscuit would you be?” You know, the sort of thing that sounds like it’s funny when you write it down knowing you’re not the one that’s going to have to deliver it to the person you hope will become your new Managing Director.
- The Expert Interview—where you’re interviewed pretty casually by the individual person who is hiring (and maybe one or two team members). In my experience this is more common. Despite my generally more scientific/statistical preference, compared to the other kind this is superior.
The problem with both is that your HR team are central, consistent, and experts in people, whereas your expert is the (erm…) expert in the specific team/skills that you’re hiring for.
To me the healthy middle ground is to invest time, and money training your experts how to interview effectively, and consistently. I’ve never had that, but I’ve started to compile an approach based on a mixture of reading, experience, and some little gems of wisdom from colleagues past, and present. This process develops a little bit more every time I interview someone, so in years to come I’m sure I will look back on this with a hint of embarrassment, but here goes.
Overall process
I’m quite standard with this. Pretty much every hiring process I’ve been through has involved these steps in some form or another.
- CV review ~15 mins
- Telephone interview ~30 mins
- (Optional) Technical test ≤2 hours
- Face-to-face interview ~1–2 hours
CV review
Honestly this may be a weakness of mine, but I find it hard to turn someone down at the CV stage. I give the benefit of the doubt a bit too much, but you need to know yourself to judge this. What to look for depends a lot on the job level you’re hiring at. At the more junior level enthusiasm can outweigh a lack of domain knowledge. When you reach the upper echelons of seniority you’re often looking for very specific skills. There are a few things I’m always on the lookout for.
- Formatting quality—you’re not hiring an author. Often you get international candidates who you shouldn’t mark down because their 2nd (or even 3rd or 4th) language isn’t perfection. If I open a CV and my editor immediately puts squiggly red underlines everywhere though, or it’s challenging to read, then I question if they’ve even bothered trying (how hard is it to click the autocorrect button?) Would you want to rely on this person to explain a problem, or document their code?
- Stretching the truth in the job history—everyone paints their CV in the best light, in the same way that every company rose-tints their Job Description. It’s a two-way street, and everyone know what’s going on. In a CV a candidate will often have a ‘technologies’ section, which can include pretty much every technology they’ve ever heard of. To me there’s an unwritten rule that this is where they express their interests, and hobbies, and they don’t have to be able to answer test questions on everything in that list. The work history section on the other hand, I expect to be completely truthful, albeit presented in the most glowing way. If I see something that is deceitful—a recent example being someone who said they had used Vue.js as a database manager??—then that is a big warning sign.
- Short stints or gaps—Short stints can often be a sign of someone who is either not very good, or not easy to pin down. This is definitely something to drill in to during both the telephone interview, and the face-to-face if they get there. Likewise with a lacuna, this is something that they should be able to explain. It’s worth mentioning that you need to be sensitive when dealing with this kind of thing. There may be reasons that the candidate does not want to share, and you have to judge if you can accept that and work around it.
- Listing a 5 meter swimming badge in the achievements section—I’ve genuinely seen this, and it was with great regret that I couldn’t hire that person.
Telephone interview structure
I have a list of personal ‘cues’ I’ve built up over time, which includes things like the order to approach an interview in, some lines which—I have been told—could be summarised as simply how to converse with a human (eg. start by asking a personal question about hobbies to relax the tone, end by describing next steps, etc.), and some technical questions if I run dry at any point.
For the overall evaluation though, I’ve settled on a 4 part approach, which you can un-memorably acronym as TPTA, if you’re that way inclined. It’s adapted from a snippet of a book I read. I believe it originated at Philips as a way of hiring electrical engineers, but I could be mistaken. It pains me that I can’t find a reference. If you recognise it, please leave a comment!
- Theoretical—this is all about their knowledge of the subject. The standards, best-practices, the community. It’s nothing to do with their past or experience, and everything to do with what they know about the field that they work in.
Example questions: Explain how x framework works? What are the advantages of x approach? How do you discover new tools? - Practical—this is about delivery. “I want a concrete example of work that you have delivered to a customer in the past, and they’d better have wept in ecstasy when they received it or you’re out!” It’s interesting to see if this matches up with their theoretical answers (ie. do they walk the walk, and talk the talk?)
Example questions: Can you describe the piece of work you’re most proud of? What was the best piece of work you delivered at company x? What challenges did you face delivering it? - Team-fit — this is an easy one to misunderstand. It’s not so much about if you think you’ll be best friends. It’s more about if they have a good attitude. Can they cope with adverse situations? How do they resolve disagreements?
Example questions: What would the perfect job look like? What’s the most challenging situation you’ve been in at work? What do you do when things go wrong? - Aspiration — this is immortalised as the classic “where do you see yourself in 5 years time?” It’s easy to dislike this kind of question, because it feels clichéd, but I’m amazed how often it turns up something unexpected, like a planned change of career direction or a misunderstanding about what the job entails.
Example questions: Where do you see yourself in 5 years time? What new skills/tools are you most interested in learning?
Technical test
I’m gonna say it, I don’t like the technical test.
We all like to think that we’re the candidate’s first choice, and how much effort is it to spend two or three hours on it anyway? The trouble is, that candidate is also doing technical tests at four other companies. And when the task said spend two or three hours, did it really mean two or three hours, or four or five? What if the other candidates are spending longer? Quickly you end up with 5 (tests) × 5 (hours), and you’re doing over half a weeks work, on top of your day job, and your personal responsibilities. I don’t think it’s fair on candidates, and I don’t think it’s the best way of assessing their skills either.
All that said, it is essential to sit with your interviewee side-by-side and talk through code together during the face-to-face interview.
The best case scenario is that they already have some open-source code on GitHub or something similar. It may not be bang up-to-date, but it can help you decide whether to bring them forward to a face-to-face.
Failing that, if you’re confident enough to progress them based on the phone interview alone, bring them in, and review some open source code together. Jump in to the React source code, or Vue, or even something completely random, and talk through it with them. Try and work out how it fits together. The key is to see if they understand, recognise patterns, and how they reason about the code.
If you do need a technical test on hand, try the following:
- Keep it small, the absolute maximum they should be spending on it is a couple of hours.
- Provide the base files for a project, otherwise they’re going to spend a significant percentage of their test time setting up their preferred coding environment/linting/webpack/test-runner etc.
- If you do use something from the internet, try and at least superficially alter it so that it’s not easy to search for.
- Get them to add a small feature to the base project you provide. Make it similar to one that already exists, to see if they’ve understood the bigger picture of the code, and to see if they refactor or not
- Get them to add tests.
You know best what matters to you when it comes to reviewing the code. I always check, when hiring at any level, if they have definitely taken the time to understand the requirements before doing the test. If any have gone completely unacknowledged that is a warning sign for me. The ability to work through the problem in a methodical way is key. A lot of time can get wasted developing the wrong thing, which can be prevented by taking a couple of minutes to master the problem first.
Face-to-face interview structure
Don’t get discouraged during the interview. A couple of times I’ve been speaking to someone and it’s started to not look very promising. If the mood turns negative, or you don’t give them the opportunity to answer questions you can fall in to a downward spiral, and you wont get a representative view of the person you’re interviewing. It’s rare that I ‘write-off’ a candidate during the interview. Always keep an open mind that—even if it’s not looking good—they might come back with something to redeem themselves if you give them the opportunity. Always remember, your job as an interviewer is to get them to present themselves at their best.
- Make them comfortable. Choose a modest meeting room. Offer a choice of drinks. Crack a joke (if you have to). An interview is a strange situation for everyone involved. In the space of one or two hours you’re both trying to judge what it would be like to spend 7+ hours a day with each other. You need to get any nervousness out the way to make best use of the time.
- Sell the company. Assume they still have a blank-canvas view (remember they are juggling several interview processes in mind). The whole opening stages of the interview form the candidate’s first-impression of your company/people. Don’t bring them in through a side entrance, or book the crummiest meeting room. You don’t want to give them a false impression, but give them the best impression that is realistic. I always start with the broadest overview of what the company does, and then hone in on what the specific project, and day-to-day looks like. Be honest about what the job involves. It’s more painful to have someone join, and leave early-on, than to keep searching.
- Delve deeper in to all the TPTA aspects from the telephone interview. Ask questions that follow on from what you discussed before (if you took notes, or have an amazing memory).
- Review their sample code side-by-side with them. I can easily spend an hour doing this bit, it’s the most valuable part of the interview for me. If they haven’t provided anything they’ve written themselves, you can set a little 10–15 minute challenge for them to do here too.
Example questions: How could you refactor this to make it better? (it’s more interesting to pick code which is quite good already, rather than something with easy issues). How would you test this? (unit, e2e, etc.) Were you intoxicated when you wrote this? (ok maybe not that one). What would you change if I added x requirement? - Meet the team. If you’ve already written them off for any reason, skip this bit. There’s no need to waste your team’s, or your interviewees time. Otherwise it’s important to show your team the respect of asking their opinion as they will have to work with this person too. It will give you a snapshot of how the team dynamic might look.
- Answer their questions.
- Describe the next steps.
- Thank them for their time, and see them out. It is a pain to book a half day for an interview, or reorganise working hours. Don’t forget that it’s a big effort from their side as well as yours.
Review
The most challenging part of the whole process comes now. Recalling everything you’ve learned—sometimes over the course of a couple of weeks—to make a final decision on the candidate.
At this point I’m often reminded of one of my favourite—if not my absolute favourite—books I’ve ever read, Thinking, Fast and Slow, by Daniel Kahneman¹. In it, he describes how he developed a system of interviewing which involved driving questions at particular characteristics, concluding a score, from 1–5, for each one, and then using a weighted system at the end to decide an overall score based on the results. He later added in an equally-weighted score for the expert interviewer’s intuition. I’ve never been in a position with enough turnover of people to test a weighting system, but I do this with the TPTA (Theoretical, Practical, Team-fit, Aspiration) system above to help me when making these kinds of decisions.
1–5 for each one, summing to a total score of between 4–20, plus my ‘intuition’ 1–5 score, multiplied by 4 so that it is equally weighted to the other parts put together.
1–5 Theoretical
+
1–5 Practical
+
1–5 Team-fit
+
1–5 Aspiration
+
1–5 (×4) My Intuition
=
8–40 Final score
The score in isolation is not so useful, seeing as you don’t know how you judge people compared to other interviewers, but it does give you a sense of the relationship between this candidate, and others that you have interviewed. It’s also useful as a kind of ‘map’ of what their strengths/weaknesses are. Looking for more of an architect/leader? You’re going to want a high theoretical score. After a manager, or have a lot of high-pressure work to deliver? Practical is going to be very important. At a more junior level team-fit/aspiration are going to carry added importance.
If after all of that you still end up with uncertainty, my best advice is to steer clear. Hiring (and then having to fire) the wrong person is more damaging than missing out on someone who wasn’t that convincing to start with. This reminds me of an article I read about a company that started offering guaranteed jobs for life. I don’t know how it worked out for them in the long run, but an unexpected short-term benefit was that they became vigilant about the people they hired, and as a result got exactly the right people in when they needed them.
I want to sign-off with a final note about an experiment I’ve recently started. I’ve always taken notes during telephone interviews to help me remember everything afterwards, and also as cues for the face-to-face. For some reason it’s only recently occurred to me to hold on to those notes, and refer to them after the hiring to see if my first-impressions present an accurate understanding of the person, and how they would fit in to the team. Once I’ve tried this with a representative number of people (this may take some time) I’ll follow up with a post about any patterns I’ve noticed.
As always, please leave comments down below. I’d be interested to hear how this fits with approaches that others are using.
Thanks for reading,
Nick
[1] Kahneman, D., Thinking, Fast and slow. (Penguin books, 2012) p. 231–232.
