What Traditional Market Research Gets Wrong

Startups beware: meaningless conclusions can be derived from surveys, focus groups, interviews and even field trials

A Case Study

I was working for a startup app company a few years ago. We had achieved reasonable but modest organic growth and were ready to accelerate our efforts. We had a strong brand image, good tech and thoughtful UI/UX. We decided to hire a company that works with startups for a fresh perspective on where our product could go and how to best achieve growth.

We had two objectives:

First, start acquiring B2C customers through social media marketing. It took some time (several months) to find the right message and target group but eventually we were acquiring high value customers for a reasonable price. Note: the process was not cheap but it was certainly something we could never have done on our own and it provided the insight we needed.

Second, find a B2B play that would help with customer acquisition and subscription model pricing. The thought behind this objective is that it would be difficult to achieve material growth one-by-one.

The company we were working with did some “market research” with their supposed Market Whisperer. It was disappointing and frustrating — mostly because they didn’t identify and solve the real problem.

We had over 1000 users but when we started trying to engage with the community, there was very weak response. Even when we offered to pay for their time.

We tried surveys, focus groups and personal interviews but ultimately the sample size was tiny because there were just not enough respondents.

The final report and analysis was, therefore, based on the weak sample size. So, the proposed recommendations were a function of the team’s gut instincts with only trivial data to back them up. Any new direction we decided to take was ultimately just going to be a shot in the dark.

This is where traditional market research goes wrong. Too many marketing companies have a set of tools/moves they make but when the tools don’t work, they don’t adapt. They simply try to jam any results into a proposed solution.

Not once did someone say, maybe this product stinks or how can we jigger it to fit another scenario. There were some fundamental changes that needed to occur in the base product before we tackled growth but nobody was talking about that.

If you can’t get a group of users engaged and passionate about what you’re doing, you’re probably doing the wrong thing.

What Are My Options?

Don’t get me wrong. Traditional marketing works in the right scenario, it’s just not a one-size-fits-all solution, and startup companies require special sizing.

For the case study described above, I agree with our step number 1; we needed a big enough pool of quality users to draw any meaningful conclusions. The process to acquire those users can feel expensive but can also be a good value.

The trouble began when we could not get enough engaged users to provide feedback. The secondary trouble came when we tried focus groups and one-on-one interviews.

The truth about some market research techniques is that people lie. If you ask 100 people if they would buy something they say, “Yes! That’s a great idea! Wish I thought of it!”. Or you get answers like, “if it only had this feature, I would definitely use it”. But in reality, when it comes time to sign on the dotted line, you would be lucky to get 10 people to commit (and honestly, 10% is actually a very good conversion rate). It’s not a malicious intent, it’s just human nature to want to make other people feel good.

Therefore, the hidden meaning in our results: we needed to dissect the underlying issue with the product that was preventing our early adopters from passionately engaging.

So, what do you do? Start by looking at your challenge through a new lens (check out my Idea/Playground article for a reframing definition).

As a founder you may need to adopt the mindset that the product might need some tweaking. I’m not talking about a complete overhaul (unless it’s really bad) but possibly just a few minor changes here or there. You will know what to do by listening to and observing your customers in an informal environment.

Go on some sales calls, watch how your customers interact with your product. Keep notes on how often certain complaints are logged. What are your most frequently asked questions? Is it intuitive? What do people not get?

Then use this feedback to inform your next product move.

Parting Notes

For your listening enjoyment: I interviewed Nicos Vekiarides who founded Attestiv on my podcast “Ideas Last”. Even though it’s early days for Nicos, he successfully found the ideal intersection between idea and playground. And although he found his initial target industry, he knows he can branch out from there.