TL;DR almost everything you learn during an interview is noise. So keep it short, focus on technical ability, and unless there is a red flag make your real decision together in 3 months.
Why does this post exist?
I keep seeing tech interviewers make the same mistake over and over. They think they are assessing something in a candidate, but all they are doing is hunting for familiar traits and creating a team of clones who are unable to innovate because they all think the same way.
This leads to wasted time for both interviewer and interviewee, a lot of confusion as to “what went wrong?” and makes the whole experience feel like a random roll of the dice.
I’d prefer to live in a tech industry where we cut out the cruft and mysticism from interviewing. If we do this, interviewing wouldn’t place such a strong demand on everybody’s time and we’ll get more diversity of thought.
In this post I’m going to outline some of the areas where I think we could collectively do better as an industry.
Why do I think I have Something to Say?
After finishing a PhD I created and managed a mathematics company for 10 years. I co-created an interview process that was known to be so tough that we used its reputation to win sales, claiming we attracted the best applied mathematicians on the planet.
I made the biggest mistake of my life when I didn’t apply due diligence to a board-level “hire”. Ultimately, I had to leave my own company to get my life back. I am well aware of the dangers of getting it wrong.
For the past 5 years I’ve been the interviewee, for companies of all sizes.
All of my experience is when there is a scarce supply of talent.
Implicit Bias and Arbitrary Rules
Intuition typically takes the form of seeing oneself (or your favourite hires) in a candidate. If you’ve ever used intuition to justify rejecting somebody, stop right now and read Thinking Fast and Slow to find out how to identify and correct your mistake.
By way of example, I recently heard a company boast that they found “a strong correlation” between those who played with Lego as a child and their “good hires”. So they added “what was your favourite childhood toy?” to their screening and only went forward with people who said “Lego”. Soon enough, all of their “good hires” loved Lego, reinforcing their belief.
This is an example of an arbitrary rule which is of no significance whatsoever. All this company does is waste the interviewee’s time (a cardinal sin), limit their pool of talent, and introduce a cultural / discriminatory bias into their workplace.
If you’re an interviewer, please go through your questionnaire and check that you haven’t included quirky questions like this. Make sure that your questions are directly assessing a quality, not a correlation. e.g. if you meant to measure curiosity, be explicit and ask the candidate for a work related example. And if being curious isn’t an absolutely critical requirement to be on your team, don’t even bother asking.
Psychoanalysis
There are a few soft things we should constantly remind ourselves:
- The person on the other side of the table might be terrified. Some of the best engineers I know fall to pieces in an interview. Getting a new job is a big deal, it’s their life, they might even be distracted about what school they would send their kids to or they might suffer from impostor syndrome. Being nervous in an interview does not imply anything about confidence, so don’t try to extrapolate. I’ve seen trembling interviewees stand up to the biggest of bullies when hired.
- Your interviewer could be conducting their first ever interview; they could be scared of how their performance will affect the next round of promotions or bonuses. They might not even be listening to your answers, just going through a tick list. It is highly likely that this will happen to you if you’re interviewing with a big company since they rotate junior staff to give them interview experience.
- The interviewer might not be particularly intelligent. I’ve had similar experiences, with one team even following up with a support request for my netlib-java library after their manager failed me for knowing more than him about mathematics.
- If you have never had your life ruined by a psychopath, don’t kid yourself into thinking that you can tell a good person from a bad person in an interview. The Psychopath Code reveals how these people can completely hide themselves in society by being charming, especially with their first impression.
The Ideal Hire
Every stage of in an interview needs to provide information that lets you decide whether to reject the candidate or to continue with them. Ruthlessly remove cruft questions and stages from your process.
I think the following formula is a winner, and minimises everybody’s time:
- cv screening for required skills and experience only
- short phone screening to weed out liars
- longer do-at-home technical test based on a realistic problem
- in person interview to discuss the test and meet the team
- trial period with a bonus regardless of outcome
But what does each stage actually mean in practice?
CV Screening
I’ve seen people rejected because their CV was too short, too long, showed .net experience, had an unflattering font, or had a big wall of buzzwords.
I’ve shown up in interviews where the agent had sent a doctored version my CV that looked nothing like mine. That means your neatly typeset 2 page CV can arrive as 5 pages of Comic Sans. And agents will force candidates to give them that wall of buzzwords, because that’s how their internal search engines work. So, interviewers, chill out about the CV.
Look for the skills and experience that your requirement needs and no more. Nobody is going to tailor their CV for your requirement. If you don’t see what you absolutely need (e.g. not enough experience with a specific technology), reject the candidate.
Another thing that is total nonsense is the thought that a candidate needs to have experience in a specific market sector. I have worked in aerospace and defence, space systems, investment banking, retail banking, commodities, telecommunications and media. Market sector doesn’t matter one iota if you’re hiring a software engineer to do software engineering, it’s (usually) anarbitrary rule that should be removed. An exception is if you’re hiring for somebody who must interface with the business and the business demands it.
Phone Screening
The phone screening is a time saver for everybody because it lets the interviewer set a minimum technical bar for the experience and skills that were identified in the CV.
The right level for the phone screening is that your ideal hire shouldn’t feel strained and can take the call on their mobile whilst walking near their office for 15 minutes, or in a coffee shop. Nobody should need to take time off work for a screening call: always be aware of the practicalities.
Interviewees, don’t assume that the person interviewing you knows anything about the role. With large companies, you might never meet this person, they probably pulled the short straw for phone screenings.
And again, interviewers, please don’t add any arbitrary rules, like expecting the candidate to ask specific questions of you. You have your script, stick to it unless you end up talking about something that is clearly well above the minimum bar. If the candidate is of sufficient stature and reputation, you might want to skip straight to the Technical Test.
Technical Test
This is the most important and error-prone part of the interview because you’re only going to be able to gather a small number of data points from the test and you need to make a decision based on it that will cost everybody a lot of valuable time (and travel expenses) if you make a bad call one way and will make you miss out on an awesome hire if you make a bad call the other way.
Do not waste this opportunity by creating a stupid annoying puzzle like Google famously do — it is borderline insulting.
Instead, take the time to come up with a problem that accurately represents what the job demands. Obfuscate something that your team recently had to work on and chop it up to be much smaller in scope. Be clear what your expectations are: if you don’t ask for tests then don’t expect them to provide tests.
Do not fall into the trap of trying to assess how the candidate solves a problem, that’s an arbitrary rule. It’s impossible to measure how in a way that stands up to scrutiny. Instead focus onwhat their answer is.
This is where you really make your interview process your own. Also use it as an opportunity to show that the work can be challenging and fun!
The most controversial part of what I’m suggesting is that I believe this should be conducted at home, not in front of the interview panel. Some people say “but they could cheat!”, yes they could, but that’s why the next stage is a face to face to talk through their answer. Allowing the candidate time and freedom to solve the problem as they wish helps them behave naturally, using their preferred tools. If they can’t complete the task then they are not suitable for the role.
And don’t you dare use one of those stupid web interface interview sites like codility that show a question and expire the submission form after a few hours. They defeat the entire point of a stress free home test. I’ve only ever done one and I spent most of the time fighting the poor internet connection.
Be respectful of the candidate’s time. The problem shouldn’t take longer than 2 or 3 hours to complete. They have to do this on their unpaid time, perhaps eating into family time or sleep, while you’re getting paid for this during office hours.
Interview
By now, the face to face interview should just be a final screening between the tech lead and the candidate, optionally including somebody who will be their peer. You’ve already accepted that the candidate is competent enough to take the role and you want to confirm that they aren’t cheating, by talking you through their answer.
If you are hiring for a senior / lead role, please do not have the candidate’s future subordinates perform one-on-one interviews. It sets an extremely bad precedent and is awkward for everybody involved.
If you’re hiring for a salaried position, you might want to add a meeting with a senior manager, so that the candidate can hear about the company culture and ask questions about their future opportunities. They are assessing you as well.
The role might demand more than technical delivery, such as the ability to deliver presentations or communicate verbally at a whiteboard, and you will want to test them on this. If that is the case, ensure that the candidate is aware of what is expected ahead of time such as preparing a short talk to give to your team on a subject area they are confident with. Do not add this unless it is a necessary part of their job. Presenting to an audience can be a nightmare scenario for some people and if it’s not in the job requirement, don’t make it an arbitrary rule.
It can be good to have a meet and greet with the team. It doesn’t need to be more than a quick introduction over a 5 minute coffee break. It helps to make the candidate feel welcome and part of the team. You might have already decided to offer them a job, but they are almost certainly interviewing elsewhere.
If you reach this stage with more than one candidate, that’s a good problem to have. Deciding between multiple candidates might come down to how the team reacted, which is basically equivalent to rolling a dice.
I completely disagree with companies that have an excessive number of on-site interviews. I know of some companies who have as many as 10 interviews spread over multiple days. You’re simply not going to be able to get top talent to take that amount of time off.
Some startups might try to add on a lunch or dinner with drinks. If this happens, you (interviewee) might want to consider having lemonade while they (the interviewers) get drunk. I used to know company founders who boasted about getting candidates drunk to find out if were hiding any secrets. If you’re a startup, be professional and save the drinks for a joining celebration.
Trial Period
How the trial is conducted, and its length, is entirely up to your company. I personally like to have weekly informal catch ups (the frequency takes the stress out) to ask how the new employee is getting on, in addition to the regular planning sessions. If I’m managing a team, I like to have 360 feedback in place and for a new employee I’d want to be getting feedback every two week. A three month trial is a reasonable chunk of time.
It’s incredibly rare that there is a disconnect after the trial period and more often than not you’ll make the decision long before the trial period is concluded.
The hardest part is failing a trial employee who just isn’t cutting it, but that’s why you’re in management. If you struggle with this kind of stuff, I highly recommend Managing Creativity in High Tech. If the employee does fail the trial, it is fair to give them enough money to not have financial problems until their next job (e.g. 2 months of salary from departure date) and will look very favourably on you as an employer.
Realities of Big Companies
The Ideal Hire is all very well and good if you’re in a small company, but if you’re in a big company you might get a call from Human Resources saying a candidate is in room 5C, the main interviewer is AWOL, and you have 2 hours with them to decide to offer a job or not. You show up and you’ve been given the wrong CV, they think it’s a Clojure job when you’d like an extra Scala developer. What do you do?
To try and address miscommunications early, always start the interview by introducing yourself and your role, what the open role is (or admit that you don’t know!), and what information you’ve been given about them. You’ll be amazed how many times this clears things up.
Assuming the interview is underway and you have the freedom to use your own interview questions, try to approximate what the ideal take-home test would have looked like, and we just all have to live with the increased noise to signal ratio that this situation introduces. One of the best approaches I’ve seen to deal with this is to just mention some isolated, heavily technical, problems that they are currently facing and discuss approaches to solve it.
Prefer questions that only require a conversation or pen and paper to answer. Don’t expect the candidate to write too much code as a result.
And the other restriction is that big companies tend to have hiring sprees followed by periods of hiring freezes, so the trial period doesn’t really work so well. Again this just adds more noise, if we knew any other ways to boost the signal we’d have done it by now.
Don’t be Ridiculous
A lot of interviewers seem to think it is OK to ask about technical details of a previous contract. This is really not appropriate. Be sensitive to trade secrets and don’t base your entire interview process on the candidate telling you what they did at previous companies. High level stuff is ok, technologies they used, but asking to draw diagrams of the architecture might be more than they are allowed to share and could lead to serious legal problems.
Turns out that interviewers don’t like it if they ask “why did you apply to our amazing company?” and you respond “your day rates are bigger than everybody else”. Personally I think it’s a ridiculous question to ask, and totally irrelevant to the role, but it’s one of the few that you have to take on the chin and flat out lie about.
And Finally…
Try not to get hung up on bad hires. Just because a bad hire did X, doesn’t mean that X is a great way to detect and avoid more bad hires. Sometimes X is just really weird. Like hiding a small muppet in your jacket (yes, really). There is no need to ask every candidate if that is something they plan on doing, it probably wasn’t the root of the problem, let’s be honest.
