Why Is It So Difficult to Assess Expertise in an Interview?
We know that software interviews are broken due to a high number of exercises, focus on remembering algorithms instead of solving skills, and a long list of etceteras that I don’t want to get into at this point.
But the fact remains that, regardless of the method, we still need to assess expertise and not just knowledge or memory. The question I’ll try to answer today is why assessing and evaluating expertise is so hard.
For the last 8 years, I have regularly interviewed people for different roles, to join my teams or to join the company I was working for. I have interviewed mostly for front-end roles, as I specialized in front-end development, but I have also interviewed for back-end and design roles.
Interviewing for the roles one has experience in it’s normally the easy part, or at least the less difficult part I should say. You know what’s important to know in your area and what is not, you know when the candidate doesn’t know something and tries to make it look like they do, you know how to smell the BS.
I think, in part, is that confidence that plays against you when you’re trying to assess expertise because it is easy to assume that the candidate knows the same technologies as you, because they answer a question that you think is hard or because the candidate knows the latest framework then the candidate must have the right experience for the role and that it’s just not true.
Defining expertise
At this point, we need to define what expertise is if it’s not knowing different technologies and frameworks or solving hard problems.
The honest answer is: I don’t really know. I have an idea, a feeling. I feel that being an experienced developer is more than that. In my book, you cannot consider yourself an experienced developer if you don’t know different technologies and frameworks or can’t solve problems but that it’s not enough.
For me, being an experienced developer is to have a natural, learned tendency to look at big problems as the parts that form them, it’s to be able to use past experience to extrapolate something reasonable (not correct, but reasonable) about a problem you hadn’t seen before, it’s to understand when you need to make a fix in a hacky way and when it’s time to dig deep and remove the issue from its root, it’s to have a tendency to look at a problem given by a stakeholder and question it until you get to the real problem and not just the solution given framed as a problem.
It’s a lot. It’s subjective. That’s what makes it hard to assess.
And this is why, without me knowing it or even intending it, interviewing people outside my main role was helpful and taught me so much.
A different approach to interviews
When I interviewed back-end developers or designers, they knew much more than I did about their role, no doubt. Intimidating as that was, that allowed me not to get bogged down with questions about frameworks or tools or particularities of their craft, even if I did I wouldn’t have been able to assess most of the answers properly.
So I decided to focus on the things I could assess, that are relevant to any role despite the craft: how do they think about problems, any problem; how would they address conflicts I was part of in the past with people on their role, what would they do in a similar situation; how are they able to explain how they think and how they would solve a problem to someone who’s smart but doesn’t know the specifics of their craft; what questions do they ask when I undoubtedly ask questions with missing details for them (because I’m more used to ask questions to front end developers with whom I share so much more context); how do they talk about past work in regards of successes and failures; how they deal with topics they don’t know or they don’t have experienced in; how they learn from their mistakes?
Another thing I started doing was asking them to explain something complex about their projects and I would ask questions because I was genuinely interested in learning their point of view. I think it is critical to be engaged in the conversation and ask as many follow-up questions as possible: if the candidate said that, for example, they migrated an important part of a legacy system, I would ask things as what problems was the legacy system creating, both for developers and for the business, how did they approach the migration, how did they decide which part to migrate first and way, did any approach failed and had to change up, how did they align with business stakeholders. And from their answers, I would continue digging up with more follow-up questions about what they did and why. At the end of the day, it is questions like these that truly help you evaluate how a candidate thinks about problems and acts accordingly, a true sign of expertise.
What I realized over time is that this approach is more beneficial than the approach I used to take when interviewing people specialized in my role. Yes, some basic technical questions to gauge if they know the basics (especially for junior roles) but approaching the interview as if it was an interview for a role I am not an expert in really helps me to assess the real experience of the candidate, which is the ultimate goal of an interview.
Conclusion
I am not claiming I have solved technical interviews, not even close. I still think that sometimes technical exercises are important, as, at the end of the day, we’re solving technical problems. But the more I approach interviews this way, the more convinced I am about moving forward with the candidate or not. Learning a new technology it’s the easy part but experience can’t be taught.
