Beyond Coding Interviews Part III
Interviewing engineers for productivity
In the first post in this series, I argued that you won’t assemble the best software engineering team with technical interviews alone, and that there is a lot of value in assembling a framework for assessing candidates’ “soft skills”.
I believe that you should aim to hire people that are smarter, more self-improving, flexible, and productive than yourself. I looked at how to assess self-improving and flexible in my previous post. Now I turn to productive.
An engineer’s productivity is shorthand for a whole range of factors:
- Raw technical skills and experience (which I won’t discuss here) which means they have the ability (but not necessarily opportunity or motivation) to produce better solutions in less time
- Ability to set themselves up to focus: managing and arranging the time they have to get as much opportunity to make progress as possible
- Ability to actually deliver the right work: being intentional about what work they tackle and in what order, and not being afraid to get their hands dirty.
- Ability to deliver work before they defocus: having a mastery of the tools they depend on, being aware of blockers and fixing them rapidly.
Great engineers strive to focus
A trait I have seen in many of the best engineers is that of focus. This comes at two levels: the ability to resist distractions to get a task completed, and also the specific awareness that other tasks are less important and won’t be able to be done, at the same time or perhaps ever.
Writing code demands a maker’s schedule, and therefore successful engineers have to develop explicit practices to give them extended periods of time without interruptions (few organizations consist entirely of ‘makers’ roles, so these practices are rarely provided for them). You might ask candidates how they organize their time, and how they interact with management, product teams, customers, or production duties. How do they minimize interrupt-driven stretches?
Nevertheless, some people focus better than others. Several of the best engineers I know are coding literally within minutes of arriving at the office. Interviewing for focus is very tough. Engineers without it can easily say they have it. Great engineers who exhibit it are often deployed to many projects so their resumes may appear to be less focused than engineers who don’t.
You can test context switching ability to some extent. You can see how a candidate responds to questions or diversions you throw in while they’re tackling a coding exercise: do they explicitly defer them to the end or get thrown off course? Look for candidates at lunch interviews bringing you back to the topic, and signs of being uncomfortable to delay. You might also look for candidates being able to describe proactive efforts to tackle the procrastination to which everyone periodically succumbs.
Great candidates are intentional about managing their time. How does the candidate set themselves goals, seek clarity around requirements, and can they talk about how and why estimates they gave were right or wrong? Do they understand planning but keep it in proportion?
Engineering is a craft, and applying technical skills depends on navigating various developer tools. You want to see signs that a candidate with any experience has taken the time to build expertise in the tools they’ve been using — not because you care per se about whether they already know the shortcuts in vim, but because it’s a concern if they’ve been using vim for a year without knowing them.
If they haven’t used git previously, but your team does, that’s not a reason not to hire them. However if they tell you that they’ve been using git for three years but can’t explain how to rebase, then you need to ask whether their productivity will continue to be similarly hampered if you hire them. The same applies even more so for SRE and other production support roles, where failing to master tools can extend downtime.
Happy with dirty hands
The most productive engineers are pragmatists and owners. They are not deterred by complex, messy existing codebases, but learn to love them as their own. They develop good judgement on when to make the simplest change, and when to invest time in adding tests, changing dependencies, or refactoring to simplify code or make future development faster or less risky. They invest in services in production, and cannot rest until they understand the root cause of production issues.
You can evaluate how well candidates can understand large existing codebases by basing an interview on one. Ask the candidate to add support for a very simple additional feature that requires them to understand how the abstractions and interfaces relate to each other. Watch as they read documentation, navigate the code and describe their deductions to you. This is also a great opportunity to see familiarity with tools and testing practices. However you need to weight a candidate’s performance by experience and language familiarity far more than for algorithms questions; as such, it tells you much less about junior candidates.
Beware the ‘architect’ mindset
The anti-pattern of being ‘happy with dirty hands’ is when engineers develop a fear of going below ten-thousand feet: a reticence to understand the gory details of codebases and a reluctance to push code. I do think this mindset correlates with candidates having “architect” roles on their resume, but it exists elsewhere too.
One exception is when engineers are promoted to ‘chief architect’ because they built a system and are the “go to” source of knowledge about it: they are often prolific coders mislabeled. Also, the role of ‘architect’ varies from company to company, and depending on the role you are hiring for, architects often do have valuable experience in navigating organizations and driving projects.
Beware the ‘research’ mindset
The anti-pattern of ‘learning to love messy codebases’ is the need to restart regularly with a blank slate. Research tends to promote this way of thinking. Many superb engineers complete (or don’t!) PhDs, or spend time working in research teams at universities, but a subset of them are demotivated by and struggle to manage the messy reality of complex production services.
Often there is a role in larger teams for individuals with a research mindset exploring a problem space and challenging the current approach, but this needs careful management.
If you’re interviewing a candidate from an academic or industrial research background, you might ask what they consider their most successful project, and whether it existed a long time: did they base other research on it? Did they release it as open source, fix issues that were reported, or generally develop a community of real users? What led them to start over?
Get unblocked quickly
Camille Fournier has a great post on how individual contributors get stuck. As she points out, an individual’s awareness of their own productivity limiters is a pre-requisite to working to improve them, and is something you should look for in interviews.
Blockers are sometimes to do with existing codebases, tools, focus, or motivation (for all of which, see above), but often they are to do with project or customer requirements or organizational structure, and they take communication skills to address.
Take initiative appropriately
Great engineers take initiative at the right times and learn the appropriate balance between asking forgiveness and permission. This is key to getting unblocked. It’s something that depends on the role and team, on the task in question, and is something you can probe in interviews through examining previous examples. Do the decisions that the candidate took make sense to you?
Attention to detail within your radius
One characteristic of many good engineers I have worked with is attention to detail. You probably already assess this in technical interviews. But attention to detail can have downsides, too.
Many engineering cultures make it easy to overstate opinions or participate in decisions even where the contribution a particular individual might add is marginal. At small startups, it’s absolutely possible to make this work. However at high growth companies this can significantly challenge engineers’ focus. It’s a common way to “get stuck”.
The strongest engineers consciously apply their attention to detail only within a certain radius. They figure out when they are the best qualified person to provide input, and when to accept what doesn’t affect them. This is an important trait to look for, or train for, at high growth companies.
Apple University specifically coaches senior individual contributors to categorize most of the work going on in the company in such a way that they don’t attempt to understand or influence it.
Similarly, Google describes a characteristic they seek in employees of “humility and ownership”:
What we care about is, when faced with a problem and you’re a member of a team, do you, at the appropriate time, step in and lead. And just as critically, do you step back and stop leading’
In this series of posts, I have unpacked some of the traits that I’ve seen in high performing software engineers and suggested ways to assess these “soft skills” in interviews. This list is far from exhaustive, but I hope it helps you be more precise about a candidate’s strengths and weaknesses and, ultimately, to hire better.