How Often Should I Conduct In-depth Research Interviews?

“The numbers don’t lie, our users don’t like our newest feature! What happened?”

“Did the engineering team perform, is the feature buggy?”

“They are world-class, we cannot fault the code they produce!”

“Your designers?”

“Our designers are absolute artists, look at how the feature looks!”

“Hmm… Does the feature solve an important problem? How many users did you speak to?”

“……”

We hear some version of this conversation all the time from executives and product managers. The company has great engineers and designers. The product is loved by their users with an excellent NPS. Then a new product feature is released and it fails to capture the imagination of users — either they do not understand it, or it was not needed in the first place. What happened? In nearly every case there wasn’t enough user research. User research is essential to making smart business decisions and can include both quantitative and qualitative research methods. Too many companies solely focus on quantitative surveys for B2B market research and lose the critical nuance and insight that an in-depth interview with a user can generate.

But how often should researchers conduct in-depth interviews for a new product feature?

As a company that believes 1:1 interviews are a cornerstone of B2B market research, we decided to investigate exactly how often researchers should be conducting them as part of their user research. Based on an internal study of over 1,000 amazing product companies, we found that a successful new product feature includes between 7 and 15 in-depth interviews every two weeks. When feature launches are preceded by fewer than 7 interviews, the feature typically has a lower CSAT/NPS than those that ran 7–15.

The interviews that were most successful included a mix of existing and potential product users, and conducting them remotely lowered the barrier to entry: Respondents were far more likely to jump on a 30-to-60-minute video call at their desk than take an afternoon off work to attend in-person user research. Further, it allowed for a more knowledgeable and often more senior respondent to participate than in-person interviews tend to.

But surveys are so fast and easy.

Yes, there is a reason that surveys are more popular than qualitative research methods like 1:1 interviews — they’re much easier to conduct. Finding actual users willing to give their time to a B2B market research project has historically involved massive cost and effort to recruit, schedule and pay such users. Thus, compared to quant surveys, the ROI of in-depth interviews with users appears poor. But is that really true? Using platforms like Respondent to recruit for and facilitate these live video interviews dramatically reduces the time and cost of qualitative research as a methodology, making the ROI of in-depth interviews competitive with quant surveys.

The most successful researchers conduct qualitative research interviews at least twice a month, stacking 4–6 interviews per week for a couple of weeks and then spending another couple weeks analyzing the findings in an effort to narrow down the main questions for future interviews. These in-depth interviews allow researchers to thoroughly explore the real pain points of users and identify the core problem a new feature sought to solve. Bringing a total understanding of that core problem to the Product, Design and Engineering teams helped them develop a feature that users would love.

At Respondent we constantly have in-depth interviews with our users and it is helping us to build the product people love. How many user interviews have you scheduled for your next product feature launch?