Introducing our innovation accounting framework
This is Part 1 of our Innovation Accounting series.
We have talked about the importance of Continuous Experimenting and working on a sustainable business model, experiment by experiment. Even more important is a way to keep track of these experiments and the progress made. Not only to see what you have learned over time, but also to account for the decisions that are made in the process and why certain things need priority over others. In his book Eric Ries calls this innovation accounting:
To improve entrepreneurial outcomes, and to hold entrepreneurs accountable, we need to focus on the boring stuff: how to measure progress, how to setup milestones, how to prioritize work. This requires a new kind of accounting, specific to startups.
Keeping track of your learnings
Usually, as a startup, you can start tracking as soon as there is a version of your product out there and some metrics are in place. This is why it is important to get an MVP out there. But your much craved hockeystick curve is usually mainly flat in the very beginning. Not because founders are not doing anything, but because looking for a working business model takes time and is hard. You usually get the most valuable insights from the most failed experiments, but this doesn’t show up anywhere in our metrics. How do you keep track of the experiments you have done, and the learnings from these experiments even before having a MVP out there?
How we do innovation accounting at NEXT
Identifying priorities and risks in your businessmodel is hard, setting up and formulating an experiment is even harder. We have found out working with a lot of startups that once you have a framework in place, it is more manageable.
For every experiment sprint or loop, we look at our canvas and decide on the things that are still risky and decide on experiments based on that. At NEXT we like to work very visually so that every team member knows what is going on and why. The starting point is a canvas. We use the Lean Canvas instead of the Business Model Canvas in early stage startups because it helps to focus on the problem and solution rather than strategy and product. We also have developed our own NEXT Canvas, which we will share with you soon.
A physical version of the canvas is always on the wall and gets updated with every single insight from experiments. The canvas helps identify risks and priorities and decide on the next experiment (or set of experiments) to validate assumptions. Once we have decided on an experiment and what it is that needs validating, we formulate an hypotheses or learning goal.
To help us formulate an experiment and to define its succes criteria upfront, we have designed this experiment card:
The card helps us to be scientific about every single experiment. It helps formulate an hypothesis and learning goal, forcing us to think about what it is that we are testing. It also helps being consistent across experiments.
How to use our experiment card
The upper row:
- Used to fill in the date of designing the experiment.
- A number for this specific experiment. This way you can attach this experiment to a canvas or other events in metrics that this experiment relates to. See it as a unique ID for this experiment
- The actual cost of the experiment, this could be time, development time or actual hard cash.
Hypotheses: A falsifiable statement. (What is it that you want tested)
Context: With which segment on the canvas does this experiment correspond. What risky assumption are you trying to validate and why? We put dots in the areas of the canvas to visually mark the segments the experiment is related to.
Learning goal; what is it that you want to learn from this experiment. (If the hypotheses is not really clear, for instance in an exploitative customer interview, this one should be even clearer.
Experiment: It is important to time-box every experiment. Set both a start and end date.
Describe the experiment; Is it an interview? A fake button experiment? Describe how you are testing the hypothesis. In relation to this, which metrics are being monitored to determine succes?
Result: When is the experiment a succes? Write this down before running it. When the experiment is done make sure that the actual numbers are updated.
Next step: Was the hypotheses validated? Did you learn anything? And what does this mean in terms of next steps. Pivot, persevere, or do another experiment because this one wasn’t really testing what you wanted to test?
To do, done
All running experiments are put up on the wall next to the canvas. Once an experiment is finished it is taken from the wall and archived. But not before we have decided on a next step, and updated the canvas or the product from the results of the experiment. The experiment cards are then archived together with the corresponding canvas ( the one just before we updated it with the results of the experiment). In this way we can see which experiments accounted for which decisions and if there are any metrics of importance that relates to them.
Avoid going around in circles; track your progress.
In the next blogpost we will dive into how we use Trello as a tool to keep track of current and past experiments and the changes of the canvases in a ‘timeline’ of sorts.
