Continuous experimenting
In the last post I talked about the importance of validated learning. Trying to validate your way to a working businessmodel. You don’t validate your businessmodel by running one single experiment. Continuously testing risky assumptions is important in every phase of your startup. Helping you to derisk the path to a successful businessmodel.
It sounds easy enough but it is much harder than you think. Your first and foremost urge is to build something really awesome. Show the world your shiny new vision as a brand new product, world domination and everything. Running experiments is something that feels counterintuitive, taking away hours of your precious time and your product. Even when you know it is valuable, even when you realize that learning before building can save much needed time and money and can lead you to more awesome products, it still feels this way. In a way it feels like homework.
You are not building a product (yet) you are building a viable businessmodel.
Experiment design
With each new experiment it is important to remember that you are not testing your product, but rather your businessmodel. Every single experiment is build to learn, not to scale. Ask why. Ask yourself what it is you want to validate. Identify you riskiest assumption and think of how to test this, in the most simple and cost effective way. Do not worry about scale, you are not building your real product yet, you are testing if what you assumed would be the case, actually is the case.
The basic structure for designing experiments looks something like this;
- Set a structure and timebox every experiment (1 or 2 week loops for instance)
- Decide on what it is you want to learn (this loop) and make this falsifiable. (true/untrue)
- Agree on a number that determines succes for this experiment and write it down beforehand.
- Design an experiment that will only test what you want to learn.
- Build and Run the experiment
- Analyse the data.
- Update your Businessmodel options with the learnings.
- Repeat.
It is important to have a structure of continuous experimenting to show learning progress towards a successful businessmodel. By the time you are ready to scale, you already will have a proven customer base, and a way to grow it into something really big.
The MVP as single proof of concept?
How does the MVP fit into this? Eric Ries talks about the MVP (or Minimum Viable Product) in his book. His definition:
“A Minimum Viable Product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.”
The term has since been used, and also heavily misused, for a lot of different things. Most of the time people wrongly refer to their first release of the product as their MVP. An MVP is not your single proof of concept that will validate your business. Rather, an MVP is an experiment that will help you validate if the solution offer you have in mind, is something that has enough value for your customer segment. I’d rather avoid the term MVP. In a way your MVP is nothing more than a solution experiment. Most probable in a series of solution experiments, that test your value proposition. Is it the first product you build to capture the value you offer to your customer? Or the experiment without a product yet to test your solution? Let’s avoid this discussion and focus on continuous experimenting. Make sure that what you are building delivers value to the customer and iterate from there.
The experiments that you run when you are still validating the problem, and finding customer segments to test with will be different from the ones that you design later on. In the early phases explorative experiments like the customer interview are important. In fact, talking to your customers will remain important throughout the development of your businessmodel. Other useful experiments later on are smoke tests and concierge models. I will explain about these experiments in following posts. Just remember that with every experiment you design, you want to learn something. Think of the easiest way to proof your hypothesis. Don’t worry about scaling. Think simple, if you have to buy customers to test, buy customers. If you have to mail people by had to mimic some algorithm, do it! You are building to learn, not to scale.
