Levelling up your “Build, Measure, Learn.”

“Build, Measure, Learn” is more than just building a first iteration and seeing how it goes. Here’s why.

I see people take the following thought process:

“We think we should build feature X”.

“Sure, let’s start by building a basic version of it with just the key features.”

“Cool, yeah and then we can test it out.”

This is a really good thing to do. This is a great way to find those “unknown unknowns”. There are always things that you can never foresee, and releasing a basic version will almost always uncover some of those. This is a great way to minimise risk, and I wish more teams/companies in the world would take this approach.

But let’s just call this Level 1. Because this is not the Build, Measure, Learn that Eric Ries was talking about in “The Lean Startup”. So if you are only at level 1, how do you get to level 2?

Level 2

Eric Ries’ approach to building products is extremely scientific. It focuses on proving/disproving hypotheses, and testing assumptions. Therefore, before we do the “Build” thing, we need to think about what we are trying to prove. Or alternatively, what is the value that you are hoping to get out of this feature/project?

I’d hope that for everything that you are building, there is a reason for it. You expect to get some form of value. So the first step is to define the value that you are hoping for.

Step 1: Define the value that you are trying to drive

Once you have determined that, you then need to find a way to measure it. Most of the time this is quite easy.

Step 2: Define your key metric

Once you have your metric, you then need to find a way to change this metric with as little work as possible. Step 3, 4 and 5 are probably familiar:

Step 3: BUILD the smallest thing that could change the metric.

Step 4: MEASURE whether it makes a difference to that metric.

Step 5: LEARN from that.

The main difference in level 2 is that you define your key metric before you build anything. This is important for several reasons:

• Firstly, it generally makes your first iteration more minimal; you are just trying to build something that changes that metric.
• It also avoids any unconscious bias that you can get from level 1 BML approach. It’s easy to release your level 1 iteration, and if something goes wrong you say “it was only an ‘MVP’”. If you define what metric you were trying to move in advance, then there is no escaping failure.
• Thirdly, and most importantly, it makes people focus on the value of software, rather than on just building things.

Is there a level 3?

Well, yes. We could talk about testing riskiest assumptions rather than just moving the key metric. But level 2 often amounts to that, so let’s keep it simple for now.

Conclusion

Why is level 2 better than level 1? Because in level 1 you are just testing whether something will go wrong. In level 2 you are testing whether it will go right. It is very possible (and common) to build software that works perfectly but doesn’t drive any value. If you take the level 1 approach then you could end up doing just that. However, if you level up your Build, Measure, Learn then you are more likely to drive the value that your feature/project was aiming for. As someone that is focused on delivering value to customers early and often, I’m much more interested in level 2.

N.B. Level 1 approach also drives mistakes in using/understanding “Minimal Viable Product”, or MVP. People often think MVP is the first iteration built in level 1. However, Eric Ries’ MVP is more like the increment that is built in level 2. Hence his comment “How minimal is MVP? Probably much more minimal than you think”.

However, I do think MVP was a bad name, and I wouldn’t be surprised if Eric Ries regrets it. Hence I don’t think it is worth picking people up about it, better to start using a different phrase instead like the “Riskiest Assumption Test”, as suggested in this post by Rik Higham. Or maybe I should make up my own: “The Value Delivery Test”?!

If you enjoyed reading please give some 👏 and/or leave a comment. For more stories like this check out my publication, The Agile Mindset.