Measuring Learning in Dollars
David Binetti

How can we extend this to include the value of invalidation?

Hi David, I love your model and have enjoyed watching you present it a couple of times now.

Could you speak a little more about how you could extend your innovation accounting model to cover the ROI on invalidation, in addition to validation of ideas?

Given that the majority of startups fail, the purpose of the pre-Product/Market Fit work is to find out as soon as possible if you will fail, so you can hit eject and escape with your bacon intact. You can then redeploy the un-wasted budget towards something more valuable. Thus, there is arguably an ROI of innovation wrapped within the work you did to invalidate your idea, and not spend millions building the wrong thing.

To reference someone much smarter than myself; I recently heard and enjoyed Astro Teller’s Stanford presentation about innovation, where he talks about the importance of killing projects as quickly as possible. To paraphrase:

You were learning what the right thing to do is.
Innovation is that 90%. It’s how much you can compress the time in that 90%, or compress the cost in that 90%.
That’s it, that’s innovation
That’s why failure matters, that’s why we send people out to fail

So I’m thinking that to really measure ROI on our customer development work, we need to count failure (invalidation) as a success, as well as validation.

Coming back to your model, and your example of “The Ugly” scenario, the PM and CFO agreed to invest $2m over the course of the year, which of course is a burn rate of $500k per quarter. In the Ugly scenario, the team finds out by end of Q2 and $1m burn that the idea is not going to be profitable. Therefore, at the extreme end, that $1m properly invested in validation has saved them $20m in an unprofitable venture.

Is that ROI on innovation as well?

The super-impactful thing here, is that that John did not fail! He successfully saved the company $19m, and that is the value of his innovation! At the very least, the early shut down saved the company $1m by not drawing out the investigation phase for another 2 quarters.

I realize as I write that this can sound somewhat naive, and is not fully formed as an idea yet. But perhaps you can take the thought and run with it?

After all, what a difference it would make to corporate innovation around the world, if more companies were to create such positive incentives for quickly and efficiently shutting down invalidated ideas!

Looking forward to your thoughts on this. Thanks in advance!

Chris Guest

Other articles I’ve written