How Validation Actually Works

Every time I talk to a group of startups, I find that many of them were taught to validate by, “Go out, share your idea with a bunch of people, and see what they think”.

By now, I’ve honestly seen over a hundred cases by now where this advice led someone to invest years of their time and often savings into a project that had “early validation” because they received “positive feedback” — and then it fails to sell or get off the ground. And it’s tragic because they could have figured it out in a month of the RIGHT kind of testing at the outset.

The short version is, you CANNOT trust (consumer) interview-based or survey-based self-reporting for validation at the value proposition level. The false positive rate is very, very high in these circumstances.

(You can do it on a successful in-market product to prioritize feature development, but that is a completely different data story.)

You NEED to run an experiment that tests for true behavior change.

The good news is, you also do NOT need money, design, or development to test for behavior change — you just need a concrete handmade prototype and to get your user to let loose with it in a realistic scenario.

The other good news is, the false negative rate is ALSO very high — the truth is that very few negatives are actual true disproof. In my career, about 5% of the projects I’ve worked with actually turn into complete dead-end negatives. Instead, understanding WHY a user rejected something is often the most fun, exciting, and delicious key to finding the pivot to the right solution.