Simple rules for defining ‘success’ or ‘failure’ for a new digital product

The most important question in product development — how do we know if our new feature has worked, or decide it has failed and we need to kill it?

Every day you make investment decisions. Should you wake up earlier to fit in a morning run? Do you choose a salad or a burger for lunch? Do you really need a new pair of shoes?

You rarely ever make a decision without some kind of expectation on the return you want. So we never build a digital product without first defining the criteria for success and failure.

Here is a super simple version of the framework we use for product success — which can be used for any digital solution which runs in internet browsers, on mobiles phones or tablets, watches, or physically on a computer.

1. Business value meets customer value. Whether you are building a full product or just one feature within it, you always need to specify how it will benefit both your business and your users. Don’t build anything before you can justify it. This seems simple enough, but it’s very easy to forget to slow down and really analyse a decision without letting bias, emotion, or hierarchical structures influence things.
2. Does it matter right now? Cut out anything that you think should be done and focus on what needs to be done. If you are simply testing out a new concept, you want to focus more on how the product is working than making it look pretty. Looks are easily upgraded. Additional features can easily be added later.
3. Is it achievable? Be realistic. Ensure you have everything you need to build, whether that be the skilled people, the time or the money. You can run out of each of these resources. Having a budget for just the first version is not the way to go. 9 out 10 times you need to go further than the first version before you get any traction.
4. If you decide yes, set the right expectations. Count up your resources and allocate them in a way that allows some flexibility, will not force you to rush (which can cause mistakes) and will hold people accountable for completing tasks. When you are building a digital product or feature, you may run into unexpected complexity or need to run a couple more tests. You will also need space to make any changes based on user feedback. Always make sure there is spare time and money just in case.
5. Choose one metric to measure success against. These should always be rooted in the first point — business or user value — and ideally be about measuring user behaviour since it is their problem you are trying to solve. Avoid vanity metrics at all costs. These are things like registered users or downloads that don’t strongly correlate to revenue and profit. What is important are active users that have direct engagement with the product or feature.
6. Once you have your metric, decide what action you will take depending on the results. For example, with Eli5’s online proposal tool ‘Sally’ we created a media section so users could add images, videos or audio to a proposal. We decided that the metric to measure success was the number of uploads — and that 5+ uploads per day, per user, was a success and less than 5 was a failure.
7. And decide from the start that if you were wrong about the product or feature, you will just kill it. Don’t let emotions get involved and become attached to a product or feature. Try not to be too proud (fear of failure), embarrassed (scared of losing respect), or even greedy (you have something to gain) to admit when something just hasn’t worked, has become unachievable or no longer provides business or user value. You’ll be grateful when you save yourself, or your client, time and money.

After using these steps to define the parameters for success or failure of your product or feature, you can massively streamline the building process — and hopefully avoid a lot of confusion, disagreements and wasted resources in your team :-)

—

If you want to read more about building digital products head over to the Eli5 publication where we regularly write for all levels of tech-expertise. We will also shortly be releasing a comprehensive e-book about our process for delivering a first version in just 3 months, rather than up to 2 years when using traditional frameworks.