Why I don’t do live coding interviews
Live coding interviews test culture fit under the guise of testing competence.
I’ve always felt a bit of a tenuous relationships with live coding interviews. I love solving challenging problems, I love talking to people about software problems and I love writing code. So, why do live coding interviews feel so gross to me?
Well, curiosity got the best of me recently and I tried a few. While I can’t speak for other engineers, I can confidently say that live coding interviews are not a good way to vet my technical abilities or to understand the value I can deliver as a member of an engineering team.
I’m not a CS grad. But, I have written code professionally for the past 7 years and held a director level position at two different companies. While I’ve taken CS classes at MIT and worked as a researcher there for a few years, I didn’t get the interview training a CS grad might. Without that practice and training, I will fail a basic coding interview 100% of the time.
As many people have talked about, there is a skill to coding in front of people — a skill that does not necessarily correlate with skill at building maintainable systems. But, a coding interview is a structured test. Given the right role, could I get good at passing that test so that I nail an interview? Certainly. But, as a professional engineer, when will I have time to ‘git gud’ at technical interviews? Even if had time, no one has ever warned me that they are going to give me a coding interview. It’s a surprise!
“A wild coding interview appears!”
Live coding isn’t something I expect to do when an engineer asks to “chat”.
For my last “chat”, I met a founder and they wanted me to meet the lead engineer. Before the call, the engineer sent me a Google Doc with the text “in case we want to jot down ideas while we chat”. Turns out that was where we were going to do a coding interview. Wild!
That was an odd case but, usually I get invited to come by an office and “chat with some people on the team”. Maybe 70% of the time, I speak with various people on the team about problems and potential solutions. 30% of the time, a wild coding interview appears! “Live coding interview” casts some spell where I get put in a tiny conference room while a guy (always a guy) takes notes as I write code with a marker. I lose the battle.
Why not let someone prepare for a live coding interview?
On the surface, I understand that you don’t want someone to prepare for a coding interview since they could look up common interview questions. But, by not telling people, you run the risk that some people are going to be more prepared than others— by chance. Coding interviews are a part of some engineering cultures and not others. That doesn’t make one more qualified than the other.
A live coding interview is a test that some candidates have practiced recently while other candidates have not practiced in YEARS. By surprising them, you favor people who have deeply practiced coding interviews in a CS undergrad program, had professional coding interview training at a bootcamp, or interviewed at bazillion companies (maybe they can’t hold a job?). If I had a friend at the company, they could give me a heads up — and then I could prepare. While a ‘surprise live coding interview’ may make sense to some, it does favor CS grads, people “in network” and people who can’t hold a job. Wild!
Live coding interviews are just as much about culture fit as the rest of the interview
As someone unfamiliar with live coding interviews, it is a space where I know I am being tested, but I don’t know what I am being graded on. Assumptions are cultural norms.
Without the grading mechanism stated and agreed upon by all parties, the interviewer can move the goal post. Or, the interviewee may optimize for something the interviewer doesn’t care about at all.
When you say ‘live coding interview’, I hear ‘we only hire CS grads’.
Here are some of my notes after my last coding interview — they have nothing to do with improving my engineering skills, and are all about improving how I perform in a live coding interview.
- First, establish what the interviewer is looking for in the coding interview
Usually when you ask directly what the interviewer cares about, they will deflect with ‘we just want to see how you think through problems’. Follow up with “how many questions will we go through in this process?” and “how much time do we have?”. The goal of these questions is to understand if the interviewer is evaluating problem solving speed, performance of the solution (n log n, n2 etc), ability to communicate as you code (a new skill!) or familiarity with the language you chose. - Ask to code in pseudocode
In my last live coding interview, I was asked to write in a real language but could not use my text editor. Then, I was asked to not worry about making the code syntactically correct. That experience was very jarring and tripped me up a lot. When asked to choose, a language, I should respond by asking if they want to test my understanding of the language or if they are just trying to make me feel comfortable. - After writing the code, try to break the code in a way visible to the interviewer
One of the challenges of live coding interviews is jumping between writing code and talking about code with the interviewer. For engineers who are not formally taught, this practice breaks their learned coding process, causing them to skip steps in live coding that they would not skip in a real coding environment. One of the steps I forgot was to try to break the code I wrote.
In conclusion
I’m far from a perfect engineer. I’m excellent at some things and absolutely terrible at others. Want to know what those things are? Talk to people I’ve worked with (or ask me!) but don’t pretend you can read between lines of code on a whiteboard.
Much like the SAT when applying for college, live coding is a structured test. I didn’t go to a school that trained me to do live coding, and so will probably fail the test. As I’ve experienced it, live coding isn’t the meritocratic space that it pretends to be. Live coding interviews weed out the people who are good at live coding interviews. Those people tend to be CS grads. If that is what you are looking for, add that to the job requirements and save everyone time. Please don’t pretend live coding is a good way to vet candidates on their technical chops.
Alternatives to live coding interviews
I’ve compiled a short list of some alternatives for testing technical competence. They aim to be inclusive of non-CS grads while being easy for an engineering team to implement fairly:
- Review a project the candidate built
- Talk through designing a system for doing ‘x’
- Review a real pull request
- Take home project (I probably won’t have time for it but could talk through it!)
- Talk to people the candidate has worked with
In short, no more live coding interviews for me. For some in depth looks at alternatives, I highly recommend reading this post by Buffer about their engineering hiring process , this one about “Programmer Moneyball” — and hope that you find one I wrote about my experience hiring engineers at Artsy helpful too.
