When Your “Good” Ideas fail, Don’t Blame the Customer. It’s You.

Here’s a familiar scenario: you’ve got a brilliant idea, you raise some funding or get approval to move forward, devote countless nights in development, and launch with some fanfare. Weeks, then months go by with disappointing results. You wonder what happened. Why aren’t customers flocking to your product? You can criticize your team or blame the insufficient marketing budget, but the reality is customers didn’t want your product the way you imagined they would.

In other words, your idea didn’t have a Product-Market Fit (PMF). If we dig deeper, the real culprit was that you deceived yourself, believing you had PMF based on your market research.

The value of Market Research depends on the data

To support your idea, you develop a market research report to provide insights and predict how customers will respond. Your report contains assumptions (unproven statements) and Other People’s Data (OPD) and your optimism bias leads you to misinterpret and miss PMF. Without market validation you ultimately end up with a market failure.

OPD is a concept coined by Alberto Savoia, a successful serial entrepreneur and author of the “The Right It.” He devalues OPD because it is “market data collected and composed by other people, other projects at other times, in other places with other methods and for other purposes,” and does not validate whether your idea will work. While not all OPD is terrible, a heavy reliance on it is misleading, creating a false sense of confidence. The way to counter the bias is by getting Your Own Data (YODA), another nod to Alberto.

To illustrate the shortcomings of OPD, let’s consider the 3D TV flop. In 2009, James Cameron released the blockbuster movie Avatar, which featured 3D effects and generated over $2B in international box office sales. Mesmerized by the movie’s success and the “Avatar Effect”, big TV brands like Panasonic and LG raced to develop the home theatre 3D experience. By 2017, Sony was the last of the major players who dropped the 3D technology from their TVs.

Why did they all fail? Two of the many reasons were:

1. Bad timing. Consumers just upgraded their television sets due to the analog-to-digital broadcast requirements released in 2009. This spending meant consumers were not interested in buying a new TV just for the 3D technology.

2. The cost of ownership was too high. The 3D viewing experience required special glasses, which added to the price, upwards of $100 per pair. Since there was no standard, every OEM had a proprietary system, which meant the 3D glasses were not interchangeable with any other TV sets. Additionally, consumers would also need to invest in other 3D-enabled equipment to create at ‘true’ 3D experience, like 3D enabled blue-ray disc players and cable boxes.

Build confidence in your data

So, the million-dollar (or potentially billion-dollar) question is, how do you spot the weakness in your data? Here’s an idea: objectively assess your data using the Market Research Scorecard (MRS) to determine a Confidence Score. Here’s how it’s done:

1. Start with a well-defined hypothesis. Every idea is predicated on a basic premise framed as a hypothesis: “if I build [this], then people will [buy] it.”
2. Score each data statement relative to the hypothesis using the six dimensions listed below. A single score on each dimension is binary, 1 or 0. The max score a statement can receive is 6.
3. Calculate the Confidence Score by taking the average of the cumulative scores.
4. Assess the Confidence Score. A confidence score below 4.5 denotes that the underlying market research data supporting your hypothesis are not sound. Further validation is needed.

The six dimensions of a statement

A Confidence Score requires you to evaluate each market research statement along the following six dimensions:

1. Source: Where did you get this data? Is this primary (YODA) or secondary (OPD)?

2. Type: Is the data quantitative or qualitative? Quantitative data, say it with numbers. Qualitative data is insight; it answers the “why” behind an observation.

3. Relevance: Is this data established as a fact or an assumption (a hope, wish, dream, expected outcome, an unproven statement)?

4. Recency: How recent is this data? Depending on your industry, you will need to specify an acceptable time range. Anything outside of this range is considered the “past.” A prediction/forecast is, by definition, an assumption.

5. Representative: Does this validate the basic premise, “if I build [it], then they will [buy] it”?

6. Significance: Does this data represent the size of the market?

Watch this short demo on how we use the Market Research Scorecard with some examples.

Here’s the key takeaway: A Confidence Score calculated from the MRS gives objectivity to the data and helps avoid confirmation bias. The MRS is not a validation tool for your idea. Instead, think of it as a litmus test to assess the integrity of your market research. Use it to spot weaknesses in your research and identify the areas you need to gather more reliable data. Remember, the most useful and reliable market research report contains primary data that is statistically significant, current, representative, and fact-based.

Extra Extra!

• Download our free Market Research Scorecard template to analyze your market research, insights, and assumptions. We invite you to leave a comment below. We want to know how you used this tool and its usefulness.
• Want to learn more about the MRS? Read our follow up post here.
• Sign up for our free MasterClass: How to Spot a Good Idea.
• If you need a discovery session, get in touch with us at connect@duoduo.studio.

About DUO STUDIO

We are Tam Lisa Danier & Hanna Phan Friend, two Design Thinking Innovation Strategists and Creative Coaches. We are future thinkers, helping companies build the next great thing through creative problem-solving.