How to do Customer Interviews — Problem Interviews

Context and why this matters

Previously, I wrote about the simple questions to ask in talking to a user. Its simplicity is great but further contexts can be helpful. Also, that series questions are going in really deep around the problems the user has (i.e why he has the problem, how often, attempted solutions etc.). That series of questions are really good for getting more details but just asking problem itself isn’t enough. There should be a structure in how to operationalize these problem interviews. For example, how do these individual problem interviews fit together in the process of validating a market.

From a clarification perspective, problem interviews are meant for discovering and validating assumptions of problems where a solution can be potentially built at a profit. Within Lean movement and Lean analytics’ framework, the problem interviews are most prominently featured within the first stages of building a startup. However, they can be done at any given point in time.

So with these questions in mind, I found a really good answer for the operationalization framework for problem discovery and validation. These are the worksheets from Croll & Yoskovitz’ Lean Analytics, which is very good.

The worksheet, practical how-to guide includes:

1. How to validate the problem, best practices, key tips
2. How to do problem interview well
3. What’s a good interview script
4. How to do post interview evaluations

Here we go diving into the details:

How to validate the problem, best practices, key tips

Finding a problem to fix: (How to validate a problem) understand the following about a given problem

1. The problem is painful enough
2. Enough people care: Solving one person’s problem is called consulting. Need a market that is homogenous within and heterogeneous between
3. They are already trying to solve it: People are dealing with it somehow. Maybe they are doing something manually, because they don’t have a better way. The current solution is the biggest competition. This is the path of least resistance
4. How aware people are of this problem: if not, what it takes to make them aware of the problem

Play book here is to talk to 15 customers to validate this hypothesis.

If you can’t even find 15 people to talk to about this issue, imagine how hard it is to sell whatever it is.

Key signs that you have stumbled across something worth solving

1. They want to pay you right away
2. They are actively trying to solve the problem in question
3. They talk a lot and ask a lot of questions, demonstrating a passion for the problem
4. They lean forward and are animated

Things to look out for:

1. They are distracted
2. They talk a lot but it’s not about the problem or issues at hand
3. Their shoulders are slumped

How are people solving the problem today?

Always look at all the people you have interviewed and understand:

1. How many people aren’t trying to solve the problem at all? If people haven’t really made an attempt to solve, you have to be very cautious about moving forward. You will have to make them aware of the problem
2. How many volunteer a solution that’s good enough? (Mismatched socks are a universal problem that no one is getting rich fixing.

Are there enough people who care about this problem? (Or understanding the market)

1. Do a top down analysis: Look at all the money in this sector then unitized to your market
2. Do a bottom up analysis: Look at the unitized price in this operation and then scale up to your capacity

What will it take to make them aware of the problem?

If subjects don’t know they have the problem but there’s good evidence that the need really exists, you will need to know how easily they will come to realize it.

1. Get a prototype early on
2. Use a paper prototype, watch how they interact with the idea without coaching
3. See if they will pay immediately
4. Watch them explain it to their friends and see if they understand how to spread the message
5. Ask for referrals to others who might care

Day in the life of the customer:

Map out everything they do around the problem from start to finish, when they do it. If mapped out well, you will understand why as well. You will identify influences.

Outlining customers’ behaviors as they go about a particular task, then aligning your activities and features with those behaviors is a good way to identify missed opportunities to improve engagement, upsell, endorse, or otherwise influence buyers.

How to do problem interview well (Problem interviews) During the empathy stage, what are the key questions to ask for a problem interview

1. Aim for face to face interviews
2. Pick a neutral location
3. Avoid recording interviews
4. Make sure you have a script

Interview scripts for the problem interview:

1. Briefly set the stage for how the interview works (highlight the goals and put the interviewee in the right frame of mind)
2. Test the customer segment by colleting demographics (ask basic questions to learn more about her and understand what market segment she represents) You’d wat to learn more about business and lifestyle of the person you are talking to
3. Set the problem context by telling a story. (Connect with the subject by walking through how we identified the problem you are hoping to solve, why you think these problems matter)
4. Test the problem: by getting the subject to rank the problems. Restate the problems you have described and ask the subject to rank them in order of importance . (Don’t go too deeply but do check if there are related problems that you didn’t touch on)
5. Test the solution: Go through each problem in the order the subject ranked them and ask the subject how she solves it today. This is a point within the interview you can identify whether you have found anything worth solving.
6. Ask for something now that you are done: You don’t want to discuss the solution at length here. Otherwise, it’d sound too much like a sales pitch. Should use a high level pitch to keep subject interested. Ideally, you’d want to the subject to agree to a solution interview with you later once you are done. These initial subjects can become your first customers if they share the problem in severity.

POST Interview evaluations: How do I know if the problem is really painful enough: (Something to be filled out post interview)

1. Did the interviewee successfully rank the problems you presented? (10 pts: The interviewee ranked the problems with strong interest; 5 pts: He couldn’t decide which problem was really painful, but was still really interested in the problems; 0 pts: Struggled with ranking the problems or spent more time talking about other problems not listed)
2. Is the interviewee actively trying to solve the problems. Or has he done so in the past?(10 pts: He’s trying to solve the problem with some cobbled together solution. That means you have struck gold; 5 pts: Spent a bit of time fixing the problem but just considers the price of doing his job. He’s not trying to fix it; 0 pts: Doesn’t really spend time tackling the problem. Ok with the status quo. It’s not a big problem)
3. Was the interviewee engaged and focused throughout the interview? (8pts: He was hanging on your every word, finishing the sentences and ignoring his smartphone; 4pts: He was interested but was at times distracted and didn’t contribute comments unless you actively solicited him; 0 pts: Tuned out. Looked at his phone, cutting the meeting short, generally detached, as if he was doing a favor by meeting you.)
4. Did the interviewee agree to a follow up meeting/interview? (8 pts: He’s demanding the solution yesterday; 4 pts: He’s ok with scheduling another meeting but suddenly his calendar is booked for the next month; 0 pts: You both realize there’s no point showing him anything in terms of a solution)
5. Did the interviewee offer to refer others to you for interviews? (4 pts: Actively suggested someone you should talk to without being asked; 2 pts: suggested others at the end, in response to the question; 0 pts: Couldn’t recommend people you should speak with (ask yourself some hard questions about whether you can reach the market at scale))
6. Did the interviewee offer to pay you immediately for the solution? (3 pts: He offered to pay for the product without being asked and named a price; 1 pt: Offered to pay for the product; 0 pt: Didn’t offer to buy and use it (if here, ask the hard question of whether you can reach the market at scale))

Analyzing interview scores: Looking at the score for all the interviewees: A score of 31 or higher is a good score. Anything under isn’t.

Look at how many have good scores vs bad scores and use that to understand whether the problem is really scalable or not.