False Positives and Product Market Fit

As we’ve mentioned before, being a lean startup is hard. Beyond the psychological pressures of trying desperately to fail at a rapid rate, there are a million ways to test a hypothesis and a million hypotheses.

How do we know which type of test to use, and when to use it?

  • Should we do a smoke test?
  • A concierge test?
  • Talk to our customers?

All we want is to build something people want. Which one of these tests will actually tell us if we have the fabled Product/Market Fit?

(This is a giant topic that we explore over many different posts. You can cut to the chase by downloading the Product/Market Fit Storyboard.)

Let’s start with the obvious…

What is Product/Market Fit?

Product/Market Fit (PMF) was a phrase coined by Marc Andreessen, a venture capitalist. As Kali Albright notes in his answer on Quora, Marc said,

Product/market fit means being in a good market with a product that can satisfy that market.

Think of a “good market” as one that is sufficiently large and growing at a rate to warrant a VC being even remotely interested.

Scaling Marketing

Sean Ellis, who is often credited with popularizing the term, had a similar context. He said that he was only able to effectively scale marketing after the company had achieved PMF and he used his Customer Development Survey to measure their readiness. Product/Market Fit was the foundation of his Startup Marketing Pyramid.

So, Product/Market Fit is about scaling.

Put simply, it’s when you stop wondering if your product actually solves a problem and you start wondering how you’re going to deal with all the people trying to buy your product and how to get even more of them to your storefront.

How do I know if I have Product/Market Fit?

Sean used his Customer Development Survey to measure Product/Market Fit. He specifically said that a 40% or higher result was necessary to have Product/Market Fit. The survey is:

How would you feel if you could no longer use [product]?
1. Very disappointed
2. Somewhat disappointed
3. Not disappointed (it isn’t really that useful)
4. N/A — I no longer use [product]
A Very Disappointed Customer

He went on to say:

If you find that over 40% of your users are saying that they would be “very disappointed” without your product, there is a great chance you can build sustainable, scalable customer acquisition growth on this “must have” product… Those that were above 40% are generally able to sustainably scale the businesses; those significantly below 40% always seem to struggle.

Asked specifically about this at the March 2010 Lean Startup Circle meetup in San Francisco, Sean repeated that the >40% result was necessary but not sufficient and we know the failings of a misinterpreted survey first hand.

StartupSquare

Our first startup in Silicon Valley, StartupSquare, had >40% on the Customer Development Survey and unequivocally did not have PMF. We had Problem/Solution Fit, but not Product/Market Fit.

Our product simply didn’t work that well. In fact, it didn’t do too much of anything.

Instead, we had measured our customer’s great burning desire for a solution, but had not succeeded in creating that solution.

Furthermore, the market was too small at that time to merit additional investment in creating a better solution. Any additional time or money spent would have been wasted on a fundamentally broken business model.

It failed both Marc Andreessen and Sean Ellis’ definitions.

Jargon

So here is our more specific and overly wordy definition:

Product/Market Fit is sufficient demand in a clearly defined marketplace for a product delivering a clearly defined value proposition to allow efficient (human or financial) capital expenditure to scale value creation.

If scaling the company results in increased costs without additional returns (either in revenue or, at the very least, company valuation) then the company is prematurely scaling prior to Product/Market Fit.

False Positives

I suspect survey.io can give a false positive result when the product is early stage enough that customers are answering based on the promise of a solution instead of the actual product.

(We admittedly have not, and will not, do the more rigorous academic research to prove this conclusively. But we only need one example to show that a false positive is possible.)

So while having <40% on Sean’s survey probably does indicate that you do not have PMF, >40% is a very strong indicator, but does not necessarily mean with certainty that you have product/market fit.

Product/Market Fit and Business Models

A single survey will never tell you if you have Product/Market Fit. -Tweet This

On our blog, we delve into this topic and how we use Product/Market Fit Storyboarding to create a roadmap of experiments to prove various hypotheses driving towards Product/Market Fit.

(If you’d like to cut to the chase, you can download the Product/Market Fit Storyboard.)

(Illustration credit for the good ones goes to Emily Chiu.)

This post was originally published here on Grasshopper Herder — Lean Startup Blog. Don’t miss a post…Subscribe!