How to not suck at product validation

Most beer-hobbyists enjoy the process of making beer as much as the beer itself. As a homebrewer myself, it takes me about 45 days for a 5-gallon batch of beer to be ready. So for me, built into the process of making beer is the opportunity cost of not getting to make a different beer than the one in progress.

Products are very similar. For every product you are working on, there is an opportunity cost of the idea that you aren’t working on. Because good products take way longer than 45 days to make, you want to minimize that opportunity cost as much as possible. The best way to minimize that cost is to get great at product validation.

A podcast called The Bootstrapped Web summarized some ideas and the process behind “The Dark Ar of Product Validation” really well. Below are my notes from it:

The key we are trying to answer: Is the idea you have in mind worth pursuing?

What is validation? The mythical unicorn of avoiding all risk of working on a new product

A common understanding of validation — Pitching an idea to someone and asking “would you pay for that”

Doing just that doesn’t make something valid.

Known frameworks for validation:

1. Lean startup — Hypothesis tested with an MVP, measured and iterated.

• http://theleanstartup.com/
• Feels most scientific.
• Requires resources that most people do not have.

2. 30x500 — Watching potential customers in their own habitat

• https://30x500.com/
• Not customer interviews, find people on the Internet who are looking for solutions.
• Forums, message boards, email lists.
• Easier to be successful if you can find real people who are already experiencing the problem.
• Sort of “going the other way”.

3. Foundation — Idea extraction i.e. drill into what the customers are telling you

• Going in with a hunch, but without your own ego to learn from customers.
• Letting the patterns come up.
• Can you get people to pay you money before you build it.

A simpler framework:

• It’s a multistep process and every step it gets “more validated”
• Each step is designed to have you take some action and get you to an answer.

Step 1: Idea (10% validated)

Key question to answer: Is this the right idea for me to explore?

Hunch, fully fleshed, whatever it is, it starts internally

Ways to have an idea:

1. Scratch your own itch
2. I don’t have the problem personally, but I see it existing in the world

Key pitfalls you might fall into:

• Just because you have an idea, doesn’t mean you are the right person to do it.
• Seeing other people have success in solving a problem and thinking you can do it.

Step 2: Doing the research and identifying competitors (20% validated)

Key Question: Who are the potential competitors & potential customers?

Art to doing research on potential competitors & potential customers

You have some tactical tools you can use here — Google, AngelList, HackerNews

What to do when you talk about your idea and someone says “have you heard of this?”.

• Elevates your education of the problem/solution.
• I’d be more worried if it is a novel, new idea.
• You want to see a mix of competitors (some solid, some crummy)
• Start getting some early ideas on messaging and customer acquisition.

Step 3: First conversations (30% validated)

Key Question: Do people have any interest in this at all?

• Survey is good to getting targeted answers to your questions.
• Goal is to get into 10–20 conversations.
• Need to be able to get on the phone and in conversations with strangers.
• How do you get to people (you don’t have an email list)?

1. 50+ hand-picked target customers who you can cold/warm email.
2. Find people on forums who are asking questions (HackerNews, etc).

• You will get rejected 9/10 times.
• Email 50+ and expect to have 5–7 conversations.
• Cold email is the best option.
• When you talk to people, ask for recommendations to other people.

Other ideas:

• Rough landing page that explain the key benefits.
• Sign up to get early access/beta.
• Reach out to people who signup.
• Can run a low-cost AdWords campaign and get to ~100 people.
• Only do the landing page if you can set that up on a weekend.
• Great way to collect email addresses.
• What you learn from the early conversations, you synthesize and send to more people.
• Reach out to the people who have said no the first time.
• Record conversations if you can — very helpful to pick up on nuances.
• Do others resonate with this problem/solution? If yes, why and or why not
• Try to get to a customer avatar.

Step 4: Building a prototype (40% validated)

Key Question: Do customers actually get value from this solution?

• Doesn’t necessary have to be created, can be done manually.
• When picking an idea, evaluate if your idea can be delivered manually.
• Try to not make it free (do whatever possible to have it be something people pay)
• Might not be possible if the prototype is very basic
• Get into the hands of 10–20 beta users so that we can watch them using it.

Step 5: Get money from customers who are paying it (50% validated)

Key Question: Are customers willing to pay for your product?

• This is the “real” validation. All steps before this have been de-risking to get to this point.
• A large portion of people who are using it for free, won’t convert.

Step 6: Sustained value (60% validated)

Key Question: Is there sustained value in your product?

• Are those first customers willing to stick around?
• Will the paying customers continue finding value in month 2 and 3 and from there on.
• Is churn/cancellation going to be an issue?
• If the product provides value in terms of saving people time or money on an on-going basis, people will want to come back to it.

Step 6: Test repeatable customer acquisition channel (70–80% validated)

Key Question: Is there a predictable revenue stream?

• Can you acquire customers in a more repeatable way?
• It’s one thing to hack and claw your way to the first few customers, but you can’t do that forever.
• Paid marketing, content marketing, distribution channels.
• Until you get to the customer acquisition step, you could very well be bootstrapped and self-funded.

Step 7: Funnel Optimization (100% validated)

Key Question: What needs to be done next to scale?

• How to get people from being strangers to get into the product, deliver value and have them pay you for it.
• You have found a marketing channels that actually work and you can now grow.
• You can now decide if you want external capital to fuel that growth.
• If you are this far along, are you still in the validation process? Yes, because you are now deciding what you need to do to scale and whether if you are even the right person for it.

Okay, that was easy :-).

In summary:

• The single most important thing is to get people on the phone.

Let’s make some beer.