Customer Interview Smells
Be on the lookout for these red flags when collecting feedback from customers
In the same way that code smells highlight underlying issues for a development team, customer feedback is littered with patterns that should be red flags for a product and research team. I have run over 100 customer interviews in the past year and have been tripped up on all of them. Without recognizing these patterns, teams can allow noise to masquerade as signal, and that’s more dangerous than doing no research at all.
“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” — Mark Twain
In the absence of any research, there is no debate about how all decisions are made — they are guesses. I can make peace with that, despite the fact it may make me cringe a bit. At least the team is clear on the level of understanding of the problem.
But in the presence of bad research, teams can convince themselves of falsehoods. Marching forward, heads down, in the wrong direction. The research can act as an anchor that drags on a team’s success as it will continuously be referenced.
Here some of the common themes I see in customer research that I take with a grain of salt. As I hear these I politely listen, nod my head, and work the conversation back on track.
1. Problems that aren’t being solved yet
This is the most controversial one which is why I am putting it first. I am of the opinion that there are no new jobs (others will disagree here). If someone isn’t trying to make progress towards something already, they won’t start. If you think otherwise, you must also be comfortable with giving your customers more work to do. And that doesn’t sit well with me. I find success when I can identify paths that the customer is already on, and focus on making that journey smoother.
A common line I hear is, as it relates to reporting functionality in products, is “it would be great if I could get a report that shows [insert some piece of information].” During an interview people want to be helpful. Sometimes they fabricate problems or ideas just so they have something to talk about. If the person isn’t already getting [some piece of information], why should I think they will start now?
A good interviewer digs in here. “What are you trying to accomplish with that information?” “How are you learning about that area today?” “Have you tried to hack together any solutions in the past to fix that?” (this last question is my favorite). I am trying to find themes where customers are struggling towards progress. If these questions fall flat, start focusing your attention elsewhere.
2. 3rd party accounts
I used to work at a large organization. Lots of structure, processes, and politics. One thing I’ve learned about large organizations is that people always want to talk about other people’s job. This is deadly when it comes to customer research. “I won’t use it, but I think someone in this other division would love this” is a toxic piece of feedback. Don’t get trapped. But do leverage this to ask for an intro to the rest of the organization.
The other scenario where this shows up frequently is when the buyer (someone more senior) is different from the user (someone more junior). It’s rare that the purchaser has a good understanding of how this product will actually get implemented day-to-day. Do not take the purchasers feedback as gospel. It’s almost certainly wrong. Make sure to get in front of the person who will be using the product.
3. Feature suggestions or preferences
People are really bad at predicting the future. But it’s easy to speak in the hypothetical which is why so many interviewees take that approach. “If [insert feature idea] was added, I would do [x]” is a common pattern that comes up in user research. Pump the brakes when you hear that.
You, as a member of the product team, are experts in solutioning. Not the customer. The customer is the expert in their problems. That’s what you care about it. Take the time to peel back the underlying reason they are making that feature suggestion. There is gold there.
4. Smoke = Fire
During customer interviews people say all sorts of things. Sometimes they ramble and often times they make stuff up. A bad jump a researcher can make is to assume that where there’s smoke there’s fire.
Just because a user is spending most of their time talking about a specific feature or flow, does not mean that this is where their biggest pain point is. If you don’t get a clear understanding of their problem and struggles, do not make the assumption that one must exist in the area they spoke about most.
Good note taking can help combat this. Often times when I reflect on a customer interview, I will remember the topic they customer talked about most. I will then conjure up a problem on this topic. Do not make this mistake. Good interview notes can ensure you don’t fall into this trap, and have evidence to fall back on.
Keep digging
The great thing about all of these patterns is that they can be protected against if you just keep digging. All of these patterns can be categorized as ultimately not understanding the job the customer is trying to accomplish. They happen because you stop questioning.
Keep going and keep digging. Don’t let them off the hook. There is gold just a few more questions away.
I have run over 100 customer interviews in the past year and the lessons above are from my first hand experience. I would love to hear other’s ideas on red flags in customer interviews.
I currently work on Product at LaunchPad Lab. If you are interested in discussing any of these topics further, do not hesitate to reach out. Follow me on Twitter @paulgonz6 or email me at paul@launchpadlab.com.
