The wrong way to interview product managers

Before joining Yieldify, I interviewed for a product role at another young start up in London. My two separate interviews with the co-founders both focused on the same question:

“What do you think about our current product?”

I was then expected to walk through the application and explain what I would change if were the product manager.

As I interviewed at more start ups, I realized this is actually quite a common approach to product manager interviews. This is especially the case if the interviewers don’t have any background in product management, such as when founders are looking for their first product manager. But this is where a lot of companies go wrong, and are likely going to hire the wrong person.

It may seem counter-intuitive, but my experience both interviewing and hiring potential employees has taught me a key lesson. If you want to build a world-class team, don’t evaluate candidates based on their ideas or their past accomplishments.

Ideas are hard to evaluate

If you ask candidates about their ideas on your current product, there is no way to evaluate how good these ideas are. The truth is, not even an experienced product manager can predict how a specific change will perform. For example, some of the most impactful AB tests I have seen were ones which my colleagues with years of experience in design said would never work. So the only objective approach would be to implement the idea and run an experiment and looking at the data, but this would be expensive.

It’s human nature to favour candidates who have similar ideas to your own. This means that if you’re using the perceived quality of ideas as an important selection criterion, you risk hiring not the best people, but those with ideas most similar to yours.

Past accomplishments are not always a good measure of future success

Another common selection method is to look at the candidate’s past accomplishments, and projecting them onto the new role. The problem with this method is that past accomplishments are often not repeatable, and hence offer limited information about how the person will perform at your company. Luck and specific circumstances are at play. For example, a lottery winner achieved an impressive ROI on their lottery ticket, but it’s unlikely they would make a great asset manager.

In one of my previous roles I achieved a 300%+ increase in monthly traffic from one of our top channels. However, this was partly due to a sub-optimal strategy before I started, and partly due to luck and timing. I can say with confidence that I would be very unlikely to replicate that accomplishment at a different company.

It’s all about the process

If you want to build a world-class product organization, you should focus on bringing in people who have the right process.

Instead of asking candidates about their thoughts on your current product, you should ask them how they would find out what to focus on and how they would test their hypotheses. Instead of simply asking what their past accomplishments were, you should ask how they achieved them, what approach they used and how they would be able to replicate the process in their new role at your company.

It is people who follow an established process that deliver value repeatedly, from one role to another.

So, what is the right process? There are many wrong approaches, but there really isn’t a single correct one. However, the good processes share a number of characteristics:

• They are iterative with continuous improvements over time
• They are driven by hypotheses
• They are based on evidence and data

Next time you’re hiring, find candidates whose work is shaped by these processes and you’ll be sure to add value to your team.