Your engineering interviews are broken (and here’s how to fix them)

The other day, #DisruptTechInterviews made its rounds on Twitter after Fiona Tay posted this tweet (Storify). It was a refreshing look at how engineers perceive your technical interview and deserves a close look if your company lives and dies by the engineers it hires or doesn’t hire.

I especially enjoyed the discussion because I’m an SF-based engineer working at a startup. But I don’t have a degree. I’ve never taken a CS course. I can’t traverse a binary tree. There’s no way I’d pass your interview, but that doesn’t stop your recruiters from spamming me. It also didn’t stop funded startups from paying me a lot of money to build their enterprise software or develop their UIs.

So it’s possible I might just be a fraud and will be unemployed an hour after publishing this. Or maybe, as Eminem once eloquently put it, “there’s a million of us just like me.” A million of us who build — or can build — software you use every day but don’t fit in your complacent, outdated interview process.

It got so bad that I even wrote a book called Hired Fast that helps those developers beat your interviews. But I’m on your side, too. I interview developers daily and understand that hiring is hard. However, I’m an engineer at heart. I’m always looking for ways to improve, so it baffles me when companies seem content with their existing interview process.

Your interviewers don’t know how to interview.

You claim hiring is important, but how much do you invest in improving that process? Are your interviewers trained by anyone on your team, much less HR? Do your interviewers know what to ask, or do they take questions off of the internet and hope the interviewee hasn’t been asked them before? If you think your interviewers do a good job, how do you know?

This is what happens when ineffective interviewers are put into decision making positions:

1. They perpetuate in-group bias.
2. They are walking legal liabilities.
3. They hire worse engineers who might make even worse interviewers.
4. They make rotten mis-hires and your team suffers for it until you fire that person, if ever.

Here’s what you can change starting today:

1. Prepare your interviewers

Besides knowing what not to ask, each interviewer should understand the context of the role and what your company is looking for. This should either be taught to new interviewers or be part of a recurring discussion, so you know your entire team is on the same page. It’s okay if the role changes over time. Here’s how to determine role context:

• What will the first 3, 6, or 12 months look like for the candidate?
• After n months, how do you determine whether or not the candidate was a fit?
• Are you looking for someone to be an expert at one thing or an expert at many things?
• Do they need to be the in-house domain expert right now or is it ok if they can ramp up to it over time?
• Are you looking for someone who can meticulously design robust systems or just someone to churn out a bunch of features?
• Will they have a lot of in-house support or do you expect them to be self-sufficient?
• Which values (engineering or personal) would contribute the most/least to on-the-job success?
• Are you looking for any traits that are valued equally as important as the candidate’s ability to code?

Knowing answers to questions like these will help your interviewers ask better questions, speed up the interview process, and avoid mistakes.

2. Ask better questions

Your interviewers might be asking questions that yield weak or overwhelmingly skewed signals. Can your questions only be answered by CS-degreed engineers? Do the questions have anything to do with what they’d be hired to do?

You need someone to fill a job position. Evaluate them in a way that coincides with how they’ll do on the job. That means no more puzzle questions, no more generic coding questions, no more programming trivia. Emulate how it’ll be working with them day-to-day.

When you design questions, you should know what a bad, good, and great answer will sound like. You can refine these over time. Eliminate questions that are too confusing to candidates (which is a time sink) or do not yield a strong enough signal.

For instance, if your engineers review code daily, give the candidate a code snippet to dissect. A weak hire might answer with X, a good hire might answer X+Y, a great hire might answer X+Y+Z + stuff your engineers didn’t even catch when designing the question.

If the context of the role implies you need someone who can contribute features across the stack with minimal support, don’t ask them if they can implement quicksort. Ask them to walk you through how they’d implement a real feature you’ve built before. Have a discussion about what considerations they’d make and why. Better yet, sit down with them for 45 minutes and do it with them. If you’re lucky, you’ll learn something new.

Decision making is easier when you have established, refined questions with clear bad/good/great answer distinctions.

You may not be able to rely on your core values, despite the fact that they’re supposed to guide hiring decisions. Here’s a realistic core value I came up with: “Our company values respecting others above all else” — you probably don’t need a core value to tell you to reject an asshole.

So let’s say 95% of interviewers are respectful during the interview. Now what? Are you a hypocrite for rejecting any of them, despite their close alignment with your core values? The reality is your engineers are either inaccurately gauging fit or outright ignoring core values when evaluating candidates.

3. Take another look at today’s talent pool

We live in a wonderful time where anyone can learn how to be a maker. In the old days, I bet you had to be the son of a blacksmith to learn how to make swords. You couldn’t just learn that sort of thing on your own, and if you were a woman you were probably discouraged from trying.

Today, people are self-taught, attend bootcamps, and hack on their WordPress blogs until they find out they have a knack for this sort of thing. I was 11 years old and made Pokémon websites when I realized I might enjoy web dev as a career. People around the world are learning how to code and it’s a wonderful thing. Not everyone gets to go to college, much less Stanford, for computer science.

You have products to ship. The last I checked, getting shit done wasn’t taught in Data Structures & Algorithms.

I understand some companies must gauge your familiarity with computer science. I’ve accepted the fact that I’ll probably never get hired at NASA at this rate. But if your company isn’t building software for spaceships or automated cars, maybe you’re doing yourself a disservice by competing with Google for the same talent pool.

Re-examine why you must ask about balancing binary trees in your interviews. Is it because success on the job requires deep understanding of the data structure? Or is it because when you and your engineers started your careers years or decades ago, you were asked the same questions and assumed it was eternal industry standard?

4. Eliminate the whiteboard

No one codes on a whiteboard, so get rid of this format. It might be an indication that you’re offering a bullshit job.

Or, for the scientifically-inclined:

No more whiteboard interviews from here on out. Allow your candidate to bring a laptop and work in their own environment or provide a laptop for interviews with common editors installed.

A symptom of the whiteboard experience is feeling nervous because there are two engineers sitting in a chair timing your answer. Unless your company mandates daily, high-intensity interval programming exercises, don’t put this kind of pressure on your candidate. Understand that some people just need to think and tinker before arriving at their answer. Encourage thoughtful, cooperative conversation in your interviews, rather than equip your interviewer with a questionnaire and no personality.

Like software development, knowing how to interview is a skill that can be improved. Devote time and (negligible sums of) money to improving this facet of your business. Invest in training to ask better questions. Invest in training to avoid unconscious bias. Spend time re-evaluating your interview process with your team and have conversations about what seems to work or not work.

And don’t just take it from me, take it from the engineers tweeting #DisruptTechInterviews — the same people you badly want to hire — who are fed up with your broken hiring.