Day One at the Lean Startup Conference 2018 — Experiment Design

For me, avid reader of The Lean Startup and most recently The Startup Way, I felt like I had a grasp of the fundamental concepts, but really wanted to dig in to some guided hands on learning with experts. So I was really looking forward to the workshops. And they did not disappoint.

I had the good fortune to attend the Mastering Experiment Design workshop led by Giff Constable and Eliot Susel.

If like me, you’ve been in situations where you just need to go. Then you go and it’s wrong. Then it’s too late. To avoid this moving forward we need to ask, “Is there anything we can do short of building the entire thing that will tell us we are going the right direction?”

To answer that question, you need to run a series of experiments.

What is the makeup of a good experiment?

  • Structured
  • Focused
  • Believable
  • Flexible
  • Compact

An experiment is a test designed to help you answer the questions “Should we do this?” or “Am I right about this?”

What is Lean?
efficiency = value produced / resources used

Value is the thing you need to learn right now. After an exercise to determine all the possible experiments you could run, pick only one experiment to run first.

It’s not really fail fast. It’s learn fast.

If you’ve anchored yourself in your vision/goals, then you should be able to determine which assumptions are the risky ones. Test the riskiest assumptions first to mitigate risk.

Part of defining the experiment is determining the measurement of value, or the outcome of the experiment. The outcome of the experiment determines which experiment to run next. Shrinking the experiment is how you improve the cycle time.

In The Startup Way, Eric Ries talks about the concepts of Innovation Accounting and metered funding. Along with how to structure experiments and measurement, the funding of running experiments using innovation accounting is an area where I really hope to learn more in Day 2.