In-context technical interviews
Last week I recapped Dave Snowden’s somewhat abstract “7 principles” of knowledge management, hinting that they may have some valuable applicability for recruiting interviews in particular. This week, I want to further flesh out this idea.
The whole purpose of recruiting interviews is to enable candidates to demonstrate the relevant skills and knowledge relevant to the role that they’re interviewing for. Yet the default mode in which many interviews are conducted in is an out-of-context one: discussing a hypothetical problem, that the candidate has never encountered before, in the abstract or in a way that’s very difficult than the way they’d tackle it on the job. This approach is particularly pervasive in technical interviews and much has been written on their adversarial nature and various other challenges.
So what would an in-context technical interview experience look like?
The first hint comes from Lou Adler (whose book I covered at length here) who recommends using an interview structure he calls “anchor and visualize”, at a high-level it’s a 2-questions interview where the first question focuses on work that the candidate had already done, and the second question focuses on a hypothetical future looking question. Both questions can then be explored further and elaborated upon using a set of follow-up questions (“SMARTe”). Looked at through the lens of staying in context, it’s easy to see why the first question is a good starting point — it is keeping candidates “in context” by asking them to talk about work they’ve already done. The second question, if applied verbatim, still suffers from being out-of-context. But if we take its intent, of seeing how the candidate deals with a new situation, and try to accomplish the same result by adding complexity/modifying some of the details of the initial situation — now we have something interesting in our hands!
The second hint comes from the folks at Stripe, who knowingly or not, sketched out a technical interviewing experience that’s mostly in-context, and best captured in this Quora post. In Stripe’s interview day no coding is done on a whiteboard. The various assignments are done on the candidate’s laptop in their own environment (again, staying in context).
Based on those two things I’d propose the following for an almost-fully in-context technical interview experience:
Take-home assignment: consisting of two parts:
- A short coding exercise in which the candidate needs to create working code from scratch — laying the groundwork for assessing the candidate’s design and implementation skills.
- A non-coding exercise asking a few comprehension questions based on a larger, existing code-base — laying the groundwork for exercises that will require a bigger code-base, enabling the candidate to become familiar with it (stay in-context) without creating an unreasonably long assignment that’d require them to write it on their own.
Candidates should be using their own laptops and environments throughout the day. I like Stipe’s overall structure for the day as a strong starting point that can then be modified / adapted to fit the unique needs of the company or the role:
- Design and implementation (90–120mins) — starting with a code review of the first part of the take-home assignment. Going deeper and adding complications and modifications from there.
- Bug squashing (45–60mins) — the code-base shared in the 2nd part of the take-home assignment should contain a failed test. In this session, the candidate will be asked to find the bug and fix it.
- Refactoring (45–60mins) — the code-base shared in the 2nd part of the take-home assignment should contain a poorly implemented section. In this session, the candidate will be asked to refactor that section.
There is one important caveat, captured in the Stripe Quora post and is worth mentioning here: There are some aspects of working out-of-context that are expected as part of the job. For example, developers are often required to maintain/support/interact with code they haven’t written themselves. I can see why some companies may choose to take a more extreme approach than using the 2nd part of the take-home assignment to assess that and will want to introduce a never-seen-before code into the interview day. I think that’s ok, but my advice would be to keep it to the one interview that’s focused on evaluating that skill and keep the rest of the day as in-context as possible.
We have a long way to go as an industry to get to this ideal, present company included, but losing out on good engineers is a luxury none of us should be willing to afford.