Exploring Software Engineering Interviews — Part 1, Rage Against the LeetCode

Why is there so much animosity towards coding interviews?

Why are we here?

Recently, I interviewed with about a dozen companies (large and small) just to see what was out there and make sure I had a good sense for my market value (most software engineers should probably do the same every 2–3 years?). As I went through the process I did the usual practice on LeetCode, along with a systems design study (which was new for me). The more that I went through the interviewing process, and the more I shared and heard from friends in the industry, the more it became apparent that software engineering interviews are a divisive topic. Like most divisive topics, I suspected interviewing to be a difficult and nuanced problem, with no “one-size-fits-all” answer.

So, here we are, at the first entry in a series that will explore software engineering interviews. We’ll be digging into questions ranging from “why do they exist?” to “how effective are they in their current state?” to “how can I prepare?” to “how should I run them as an employer?”.

If you spend any time looking up coding interviews on the various media platforms, you’ll find a plethora of preparation material and a healthy (unhealthy?) chunk of animosity. The latter, it seems, is popularly stated as “coding interviews are broken”. (I’ve even seen the words “cruel” and “abusive” used by those particularly peeved).

There is one key point that makes up this objection, and I think it is best phrased like so:

Coding interviews are not representative of the work you’re being hired to do

That’s the key idea we’ll explore in the first entry of the series.

Coding interviews are broken?

There are a few underlying ideas behind that main point. In no particular order:

• Coding interviews do not have the same scope as typical programming work. They’re almost always pure functions (no side effects, idempotent) with an incredibly finite set of constraints. Typical software projects are rarely either of those things.
• Coding interviews focus on data structures and algorithms which rarely if ever, come up in typical programming work. When was the last time you used a linked list or heap in your day job?
• Coding interviews don’t examine much of the pragmatic side of development: how are your code reviews, how well do you design software, how are you at debugging, etc., etc.
• Coding interviews aren’t usually in a realistic dev environment (whiteboard, plain text editor, etc.)
• and so on…

These objections can loosely be bundled into two classes. The first is “skills you must-have for a job that you don’t use in the interview”. The second is “skills you must have for coding interviews that you don’t use in your job”.

Let’s briefly consider the first class. Obviously, it isn’t possible, in forty-five minutes (or an hour), to completely emulate a role that owns projects which take weeks or months to complete while interacting with potentially dozens of stakeholders, users, designers, etc. This means you’re forced to pick the skills that are most important to you as an employer to evaluate in the limited time you have. I’ll refrain from diving deeper into this line of reasoning, because for our purposes it’s enough to say that this class of objections is legitimate, and that it will always exist (because interviews are limited by time, and will never be able to cover the entire set of skills you need to be successful as a software engineer). Based on what I’ve observed, the best way to solve this is to have multiple types of interviews (e.g. systems design, debugging, take home exercises, pair programming, etc.) to get sufficient depth and breadth of signal on the candidate.

What about the second class, “skills you must have for coding interviews that you don’t use in your day job”? First, consider the question: what is the goal of a coding interview? At the highest level, the goal of a coding interview is to answer the question, “should we hire this candidate?” More specifically, the goal of the coding interview is to evaluate the skills deemed critical to success in the given role. What are those skills? Based on internal rubrics (for coding interview performance evaluation) from software companies big and small, the skills can essentially be boiled down to these three: communication (specifically of low-level technical ideas), problem solving (understanding the problem statement, identifying key constraints/edge cases, conceiving of solutions that are correct, verifying/debugging those solutions), and coding (implementing the proposed solution, comfort expressing oneself in code, correctness). It is worth noting here that “coding” as a skill includes much more than what is required to code a solution to an interview problem, but that falls under the category of the first class of objections and thus we’ll leave it be.

These skills are what the coding interview seeks to evaluate (for the purpose of this entry we’ll forgo discussing other types of interviews in more depth). The medium by which you assess these abilities (within the bounds of a 45–60 minute interview) is by offering some coding challenge that gives the candidate an opportunity to demonstrate them. However, there are some problems that provide much better opportunities than others.

There are two characteristics of coding problems that are poor choices for interviews:

• any problem that requires knowledge of something arcane; that is, a data type or algorithm that can nearly be described as purely academic in nature
• any problem that requires a “trick” that’s unreasonable to expect a candidate to identify if it’s their first time seeing the problem

Asking a candidate a problem with these traits will only tell you two things. First, does the candidate have the arcane knowledge or know the trick already? Second, if they don’t, how do they handle the situation? The former is almost useless signal. If the candidate could go acquire that knowledge or learn that trick with a quick google search, and thereafter be successful, then we’re learning nothing about the core skills we’re trying to assess. The latter is potentially interesting but also skirts the core skills we want to evaluate.

Therefore, the second class of objection also has legitimacy. Interviewers should do their due diligence when selecting coding questions from their company’s pool to ensure that the questions they’re asking to avoid those two pitfalls. It’s in the interviewer’s and the candidate’s best interest to pick questions that maximize the quality and quantity of the signal received around the core skills being evaluated. How frustrating would it be to show up to an interview and be asked to traverse a binary tree using constant space (Morris traversal), or Union Find, or Dijkstra’s? How strong is the signal you’ll get as an interviewer asking a question whose answer was designed by a dude named Morris over forty years ago?

An aside

I’ve frequently heard (both in popular content criticizing coding interviews, and in personal conversations) that coding interviews require “grinding leetcode.com” and/or “a bunch of memorization”.

If all it took to become proficient in these types of coding interviews was time, then it stands to reason that anyone who invests the appropriate amount of time would succeed. However, I personally know many, many people who have done hundreds of practice problems and are still struggling to pass these interviews (or are abjectly failing). Anecdotally, that group outnumbers the people I know who have been successful by at least a factor of two. If practicing or memorizing was all it took, wouldn’t people who have invested hundreds of hours pass their interviews? At the very least, we can conclude that success in these interviews requires more than simply memorization and time spent practicing.

What about the baby?

Does this mean that coding interviews really are broken? Should we burn it all down and start from scratch? Is there a baby in the bathwater?

We already discussed the skills evaluated during a coding interview: communication, problem-solving, and coding. They are arguably the most fundamental skills for a software developer. They are also incredibly reasonable skills to base hiring decisions on. Would you want to work with (or hire) a developer who has weak communication, problem solving, or coding skills? These skills are worth evaluating, and good coding problems do a terrific job providing insight.

A high-quality coding problem:

• is easy to understand (while it should require clarifying questions, the problem statement should in no way be confusing, tricky, or misleading)
• is difficult to solve (easy problems provide little insight into any of the skills we’re evaluating)
• uses fundamental data structures and algorithms (where fundamental is mostly determined by the role the candidate is applying to)
• bonus: is conceptually relevant to the types of projects you’d do on the job

These three traits maximize the opportunity to evaluate how well a candidate communicates, how well they solve problems, and how well they write code. They also maximize a candidate’s opportunity to succeed, without lowering the bar you’ve set as a company.

Note: “fundamental” depends on the company/role. Generically, this may include things like lists (array and linked), maps, trees, graphs, etc., but may omit things like dynamic programming, bit manipulation, named algorithms, etc. For example, nearly all of facebook.com’s data is modeled as a graph. It would be reasonable (expected, maybe?) for a company like Meta to ask a graph-based coding question. However, not every company has that type of data model, so graphs are likely not “fundamental” for most companies.

TLDR;

In short, some of the rage against coding interviews is well-founded. Questions that require some arbitrary knowledge that you’d never need in your job or require some trick that you’ll likely never figure out on your own make it hard for candidates to succeed and provide a poor opportunity for the company to evaluate the candidate. Unfortunately for us, the largest tech companies can afford those shortcomings by filtering out strong candidates, simply due to the unbelievable volume of candidates who want to work there.

On the other hand, good coding interviews maximize both a candidate’s opportunity to succeed and the quality of the insight for the interviewer. They provide signals on how well a candidate communicates technical ideas, how well they solve ambiguous problems, and how well they convert solutions to code.

The takeaways

To address the current pain of coding interviews (at which the rage is legitimately directed) companies (of any size) must be more discerning in how they build their question pool, and interviewers ought to be equally discerning in selecting a question to ask. The only way to effect this change is to start with yourself. Do you have influence over-engineering interviews at your company? Are you an interviewer? If not, should you become one? If you are, what types of questions are you asking? Are you focusing on maximizing both the candidate’s opportunity for success and your opportunity to collect meaningful insight on the core skills of a software engineer? Should you encourage your company to include more types of interviews to improve the breadth of signal you’re getting?

Postscript

It’s worth noting that the answer to the question “what is the most effective interview strategy?” is going to vary from employer to employer. Large companies have an ocean of candidates and can afford a plethora of false negatives (which frequently come with the types of coding questions they ask). Smaller companies have a finite pool of candidates and must be diligent in minimizing false negatives. Smaller companies also may bias more towards system design interviews, because their software systems are frequently not yet well established.

Sign up for our Free Weekly Newsletter. Fresh content from developers all over the world, fresh and direct from the field! Don’t forget to follow our publication The Dev Project.