No feedback? No problem! —
Methods for unintentional user feedback
Usually when we run into a dilemma, our instinct as product managers tells us stuff like: “let’s get out of the building,” “let’s ask the user,” “we should build a focus group” etc. this is a good instinct!.
Direct user feedback is useful: testing hypotheses and exploring features by asking questions in person, sending questions as surveys and building focus groups, gives users a chance to speak their mind — hopefully giving us low hanging fruit to develop.
But how can we deal with not being able to contact the user? This is a situation that most B2C product managers are not used to. I come across this challenge in my current position — creating an app for bar owners. The reason is that my company was purchased by another company, which is under strict monopoly laws, not allowing us to contact the end-user or offer incentives (such as an amazon gift card for instance).
Summing it up so far: Direct customer feedback is unattainable, so what can we do?Most product managers would use these proven methods:
- A/B testing — An oldie, but a goodie. No need to go much deeper, start with an assumption, run a test with a feature built based on said assumption, monitor a control group using the current version, and voilà!, The feedback will tell you what situation is preferable, A or B.
- Test a group similar to your early adopters — If your main users are out of reach, look for the next best possible thing: a group similar enough to your first users, that have yet to use your product, and are accessible for interviews and usability testing.
This method was actually quite helpful, as I contacted several bar owners in my area. The insights were interesting, but quite local in nature. Another important tip is targeting first adopters. This is crucial in a low tech environment (for example the nightlife environment), otherwise you might find it hard to extract feature ideas from your interviewees.
- Screen recordings — Not much to say here, grab some popcorn, track a specific feature, and just try to get into the usability testing vibe. It might take some time until you hit something useful.
Sadly, in my current position we were not able to use this method. My apps infrastructure is a mobile hybrid app based on Ionic, an infrastructure that does not allow us to use all the good on shelf products for screen recordings.
Losing the traditional methods of non-direct feedback led us to try more creative, out-of-the-box-thinking feedback methods:
- Reading support tickets — You won’t believe the gold you can find there (after you sift through all the angry stuff). Users that care about the product are the ones that complain the most when something unexpected happens (UI issue, data issues, crashes etc). With enough reading, you can hit the jackpot — a great suggestion for an improvement or new feature.
For example, one of our valued users from North America wrote that he needed to produce a specific weekly report, but the page he wanted to use for this was inaccessible due to a bug. We never intended the app to be used in that manner. This was a basis for a new possible feature: Generating inventory reports for users.
- Internal usability testing (and guerrilla usability testing) — Releasing a half-baked product with a problematic flow will make your users feel stuck and frustrated. And that is just awful. It’s really simple, grab a co-worker that never saw your app brief them on their role (i.e. you are a driver, show me how you use the Google Maps app to get home from the office) and let them point out what they cant do, or get frustrated doing, track their reaction!
Can’t find a co-worker? Go guerrilla. Go to a public place with a demographic that the app targets, offer free coffee, and test your flow on strangers. It’s not easy at first, but getting sh*t done rarely is.
We implemented this on a grand-scale, basically inviting the entire company to a “Beer & Testing” hour. It worked perfectly — we got 40 different pieces of usability feedback in one hour, found a couple of bugs, and had a beer — this is the fun way to test a product without real users.
- Track sensitive users more than alphas — This is an easy one: Noticed a customer complaint? Track that user. Know that there is a data issue for a specific demographic of users? Track those users. Why, you ask? This would be a good indication on what data is important, what flows are critical and what causes churn.
For example, If group of users that have a bug in a specific feature stop using the app — they actually validate this feature is worth. You’ll also want to track the number of complaints coming per feature, the more important the feature the more complaints would arise. Don’t forget: you’ll also need a test group for comparison, so big blocker bugs might not be the best way to test your sensitive users.
- MVP ghost buttons — Warning: this might piss off your alpha users. Not sure about the development of a certain heavy feature? Want to check if the users actually want it without speaking to them? MVP it via ghost buttons. With the right copy, you can get a genuine estimation of the users who want the feature, by counting the number of clicks on the ghost button.
For example, instead of developing a friend referral system that nobody might use for your product, place a friend referral button at the right place, and track conversion.
We’ve placed several of these in our bar-owner app, and definitely got a lot of interest regarding specific buttons, therefore validating the need for a certain big feature, without talking to a single user.
Not getting direct feedback from your users is not a death sentence for getting user opinions on features, it might be even helpful at times. There is an upside to indirect feedback: The user is acting “naturally,” meaning their activity is not affected by the test. There is no need to ignore pretence and the PM collecting insights is not affecting the results. So this is an obvious upside in using “sneaky” user feedback methods.
