“How do you know if your brilliant new feature has worked out? How do you know if it has failed and you need to kill it?” I asked.
“The best teams set exit criteria” Nick Coster replied. Nick, the co-founder of Brainmates, one of Australia’s top product manager trainers, went on to explain that the best teams he has worked with set their criteria upfront. What does success or failure for their new product or new product feature look like?
Let’s unpack this statement because within it there is so much gold to be had. Let’s understand why doing this is essential and also how you can implement it for your teams.
5 Types of Success Criteria for New Product
Setting your success criteria at the outset has a range of benefits. These are five things to consider:
- Focuses everyone on what matters. Having a clearly defined criteria for what will lead to success will focus everyone on the most important activities that are needed in order to make the feature a success. It will help cut out all those activities that seem like they “should” be done but really aren’t necessary or aren’t necessary right now, in order to gauge the success of your new feature.
- Avoid emotion perpetuating in a product or feature. One of the greatest drivers of poor decisions is emotion (just ask leading investors like Buffett or Howard Marks). You fall in love with the feature, you don’t want the feature to fail because you fear job loss, you desperately want it to succeed to prove something to the world, you’re greedy because some financial incentive is perceived to be there. Setting clear criteria as early as possible removes the emotion.
- Ensure what you’re doing is achievable, useful, or valuable. Taking the time to determine your success or failure criteria at the outset usually, by its very nature, causes you to consider how the feature will be valuable to the business, be useful to customers and be achievable.
- Kill bad ideas ahead of time. Thinking about the criteria might lead you to kill the idea before you waste time and money on it. You may realize that, in order to be successful, the activities you need or the success criteria is unachievable or provides no value to the business.
- Alignment on expectations. You’re most likely operating with a team, whether you’re in a company with thousands of people working on product or you’re in a startup. Setting expectations at the outset for new products has huge benefits that will help avoid arguments and difficulties further down the track, when things have or haven’t gone as planned.
Lastly, you wouldn’t make an investment in stock or property without some kind of expectation of the return you want. So don’t invest in your product without an idea of what the return will be.
7 Ways to Apply Success Criteria
Now that you understand the benefits of setting success criteria for new features and new products, you’ll be looking at how to apply this. We’ve even made a template which you can get here, that will help you when setting your criteria. Here are some useful guidelines to use in order to get the most out of setting success criteria:
- Choose only one metric. This makes it easier for teams to rally around, easier to understand, and easier to focus on the activities that are important (and not so important). The one caveat to choosing one metric is that you may need a counter-metric or two.
- Choose a metric as closely aligned as possible to both customer success and business success. It’s OK to spend more time here than you think is needed to really ensure the metric you are choosing is as close to measuring whether a customer is getting value and the business is or will receive value. If you aren’t addressing both then be careful, it may be that you’re measuring a vanity metric or setting yourself up for a product feature that isn’t going to move the needle.
- Make the metric specific to the feature/product you are creating. Personally, most of my mistakes have been from using general metrics where a more specific metric, specific to my exact product, would have better allowed me to set success criteria. For instance, active users is often touted as a way to set success criteria but if you haven’t previously had clarity on what an active user means to your product then active users is just a vanity number.
- Change the metric if you need. If you discover that the metric you have chosen isn’t the best one you could be using to judge success or failure, then change it. Just don’t change it to make it easier to pass so you feel good.
- Make it a measure of people and their behavior. Usually products are about changing behavior in some way or taking action in some way for people. So the closer your metric is to people and their behavior the more likely it is a better measure of success.
- Write out what you will do for different results. For example, if 20 messages are sent with this new feature by each customer then it is a success. If only 5 messages are sent each week then it is a failure.
- Use a hypothesis if you don’t have the data. Using a hypothesis driven approach will let you keep moving and keep things measurable. There are also more benefits to a hypothesis-driven approach.
Now you know how and why to set success criteria for your new feature or product, so when you and your team are celebrating product success, you’ll know it’s for the right reasons.
Originally published at https://www.terem.com.au on May 23, 2019.