Here’s The Easiest Way To Get The Best Customer Feedback

When it comes to customer feedback, the old adage rings true: actions speak louder than words.

This may seem obvious, but a lot of startup founders and company executives have certain misconceptions when it comes to how feedback should be obtained. Namely, they center their entire feedback strategy around focus groups. Essentially, they round up random segments of people into a conference room with free food, ask them their opinions about various important questions, and walk away with answers that are neither reliable nor valuable.

This is an antiquated — not to mention unproductive and expensive — approach to obtaining customer feedback.

Here’s why.

Talk is cheap.

That’s another adage that holds truth to it — and for good reason.

For one thing, words really don’t tell you very much. Words only show one’s attitude — and there’s a big difference between attitude and behavior. The latter is the real thing of value when it comes to feedback. You can easily fake an attitude.

Because of the artificial nature of the experience, people in focus groups will often tell you either 1) what they think you want to hear, or 2) what they aspire toward in their minds but don’t actually do in real life. If a user doesn’t have any skin in the game, especially — or, if they don’t have anything to risk by being dishonest — their words might not have any connection to their true feelings.

Ultimately, observing users in a focus group is like watching animals in a zoo: their behavior is going to be fundamentally different than it would be in a more natural environment where real time, attention, and money is on the line.

What’s more? When you’re running a focus group, it’s really easy to simply ignore the findings. When I was working at Microsoft, we conducted dozens of focus groups to acquire feedback around Bing’s social and search features. What I found was this: when the findings disagreed with whatever biases or preconceptions our team already had, the findings were generally discarded. Someone on the team would claim that the methodology was flawed, or that the sample of users was too biased.

We were able to do that, as it happens, because the feedback itself was cheap; the words were easy to discredit and ignore in a way that real usage data simply cannot be.

Even when they’re being honest, people are simply not all that skilled or capable at expressing their true feelings. It’s a difficult thing to do, even for the most articulate people. So to truly bridge the gap between attitude and behavior — which is why companies run focus groups in the first place — we need more than just words.

Rather than focusing solely on what customers say they want, it’s in a company’s best interest to pay more attention to how customers act.

The problem is, this is more difficult than it might sound. In order to effectively gauge a customer’s feedback through their actions, you need to do three things:

1. Analyze publicly available purchase & usage behaviors.

First, in absence of your own available data, you have to look at the high-level behavioral data. This includes purchase history, search history, usage history, etc. of where people are spending time and money (Amazon, App Store, etc.). This will allow you to determine who the spenders in a given space are, along with what they’re spending their money on. You can also determine what they’re saying by extracting data from publicly available sources, such as verified Amazon reviews.

If you can do this at scale — analyzing thousands of reviews and data sets — you’ll have a more accurate and complete picture regarding what is and isn’t important to people in your category. As the most basic tactic, you can run word frequency analysis of words mentioned in positive and negative reviews to figure out what parameters are important to your users. Taking it one step further, you can layer on sentiment analysis (if you don’t have coding experience try Aylien, a Google Sheets add-on that lets you do this).

If you want to get more sophisticated with analysis of behavioral data, you can buy anonymized transaction data from credit card companies, like Visa, MasterCard, and AMEX.

2. Drive real ads to a simulated experience.

Next — and this is a regular practice in the gaming industry — is to run real ads to a fake page that mimics the real App Store to see what kind of people interact with it and how. This amounts to creating an environment that mimics and encourages real world behavior where you’re selling a fake product or offering. On the web, this can be done by driving ad traffic to a landing page. In practice, no matter the platform, you’re showing real ads to real people and determining how many are sufficiently intrigued to click on your ad and ultimately hit your fake “Buy” button.

There are resources available to help you with this, too. At Glu — the mobile gaming company that acquired my startup last year — we’ve used StoreMaven andSplitMetrics to set up simulated App Store pages. These allow us to measure conversions, along with what variations of our messaging and branding prove most effective.

For example, in the early stages of our development process, when we’re still determining how to position a game in the market, we use these tools to check which themes of the same gameplay resonate most with users — military, sci-fi, sports, etc. The information we collect is relevant and actionable. We’ve found that by testing variations of everything from Name, to Icon, to Screenshots, our conversion rates on a product page can go up by 5X.

3. Launch to a small segment before going global.

Finally, the last tactic you can employ is releasing a small batch of your product to a subset of your audience. In mobile gaming, this is common and is called “soft launching.” On the web, this is often called “flighting.” It’s the best way to obtain valuable and honest feedback through customer actions.

When launching a game, we always soft launch into a smaller country with a product that’s still a work in progress but has the features we want feedback on. The goal is to identify which features resonate the strongest with the most people before you launch a more traditional, global PR campaign. On the web, companies can do this by flighting a new feature to a subset of users, and obtaining real action data from them regarding whether they find the new feature useful. If the flight to 1% or 10% of the audience improves key KPI’s, then the feature can be ramped up to the entire audience.

The key is, for these tactics to work — no matter what industry you’re in — you have to have a set process and know what you’re doing ahead of time.

There is a big opportunity for companies outside of Tech and Gaming — in fields like CPG — to adopt these lean feedback tactics. Conducting these campaigns effectively necessitates not only a new mindset, but also some serious planning. You have to create assets, invest time and money into analytics, and carefully map out different variations of what you want to test. If you’re not prepared internally, and if you don’t have metrics in place to analyze and test your findings, you won’t be successful.

This sort of prudence is particularly important for smaller companies — teams that don’t have millions to spend on user testing, or at least not the sort of chest that companies like Microsoft, Apple, or Google possess. Those are companies who can afford to launch something new and push it through brute force to prominence.

You likely won’t have that luxury, so it’s critical to plan accordingly. One thing that I will note is that getting started is cheaper than you might think. While a traditional focus group can cost $10K-20K or more, running Facebook ads to a landing page with a large enough sample to be statistically significant can be done for a few hundred bucks.

At the end of the day, though, the care and forethought required of obtaining feedback through customer behavior as opposed to attitude is worth it.

Certainly, it’s more economical than investing in focus groups.

Instead of spending money and time acquiring information that is itself unreliable, invest that effort in obtaining feedback centered around customer behaviors. The information you collect will provide you insight that is quantifiably more valuable.

After all, the old adages hold water for a reason.