3 Good and Bad Questions for Collecting User Feedback

(I initially wrote this a few years back on my old blog. Reposting here because, well, I still have the same views).

My job wouldn’t exist without user feedback. I am constantly trying to improve on the type of questions I ask, but so far I have a definitive list for“good to ask” and “never ask”. This is specifically referring to interviewing users who are familiar with the platform.

Never ask

1. “How can we improve our product?” Almost every user will have an answer to this one. Power users could talk to me about these for hours. But as a PM, am I really running short on product ideas? This question could be a great ice-breaker, but I wouldn’t take many notes. From my experience, focusing on problems users are facing is a more rewarding tactic. Once I have a good understanding of their problems I can go back to my desk and do my homework (think about solutions). Often users will suggest ideas without themselves realizing what it is they’re trying to solve. This is not to say that users ideas aren’t good. The truth is that I, as a PM, spend most of my day in this “idea” mindset, while they do not. It’s unfair to expect anyone else to have the same depth of understanding the platform, the market, the problem, etc. However, this is an excellent question to ask users offline, in a feedback forum, for example. This calls for a separate blog post…
2. “We’re thinking about implementing feature x. Would you use it? ” Of course they would. I rarely had people complain to me about new features. A better version of this would be trying to figure out if my user is struggling with whatever feature x will be solving.
3. “How do we compare to our competitor x?” Honestly, this is a difficult question. I often have a hard time answering it myself, and I think about this stuff on daily basis. Rather, I like to ask what it is they like about competitor x.

Good to ask

1. “Describe your day.” This question varies a bit depending on the product we’re talking about, but the goal remains the same: figuring out where in this person’s life does that product fit. What do they do just before or just after they use our product. What triggers them to come back to us. What other tools are they using throughout the day (so, for example, if all your users use a complimentary product X, you might want to add a variant of X’s core feature to your product next). This is also an excellent source for user acquisition strategies!
2. I ask them to walk me through our core use case. For example, at Threadflip, users list their own fashion items for sale. I like to ask them to describe, in detail, how they go about uploading an item. I ask what do they do just before uploading (and how it is that they decide to list something for sale), details about the listing process, and what happens afterwards. My goal here is to listen carefully and note any “unexpected” steps. And there are guaranteed to be at least a few.
3. “What bugs you ALL the time?” I like to close with this question. It’s a difficult one. Most of the time people just can’t think of anything, and that’s fine. But the times when I do elicit a response, most surprise me. It’s rare that they bring up a core feature here. It’s usually the smallest things, enhancements that were patched quickly, or a glitch in the notification system, or an email that they receive, etc. All low effort stuff from the engineering perspective, but it’s eye opening that this is what they bring up when they could have picked virtually anything!