How do you measure the success of a product feature?

From time to time I have been asked about how to measure a product feature's success. It sounds like a trivial question, but it does not.

A product feature daily usage. Source: amplitude.com

The first important moment that you need to take into account is that to answer this question you need to distinguish between product metrics and business model metrics.

Probably some of you get known with the AARRR framework by Dave McClure. AARRR stands for Acquisition, Activation, Retention, Referral, and Revenue.

The AARRR framework is good when you want to assess your business model and probably find some points of growth. But this framework is too high-level to be applied to the product feature level. So we need to find something more concrete.

If you want to find some framework specifically designed for improving UX you could probably need to look into the HEART framework by Google. HEART stands for Happiness, Engagement, Adoption, Retention, and Task Success.

Indeed, the HEART framework is way better if you want to start to assess product features, but it is still high-level and does not provide details about what to measure specifically and how to evaluate a specific product feature.

If you google `How to measure the success of a product feature` you can get millions of pages, but in my opinion, the most useful of them is an analytical article from Amplitude written in 2016:

Google SERP

In short, Amplitude recommends doing product feature analysis via the following steps:

• Measure basic usage of the new feature
• Dig deeper into event properties to look for patterns
• Understand what users are doing right before using the feature
• Build a behavioral cohort of people who used the feature
• Analyze the impact of the new feature on retention
• Measure the impact of the new feature on your key conversion funnels
• Measure the impact of your new feature on engagement

What is more important is that Amplitude, at the end of their guide, reminds all readers: you don’t have causation yet!

Nevertheless, I think there is still room for improvement in answering this critical question that I asked at the beginning of this post.

That’s why below I will introduce you to my own framework for assessing product features.

Each item in the list below is a dedicated article with some insights about how to calculate metrics and how to interpret them.

1. how popular is this feature among Weekly/Daily Active Users (popularity)?
2. how regularly are users using this feature per week (habit)?
3. how often are users using this feature per day (usage)?
4. what is the impact on retention rate (value)?
   — An enhanced version of the analysis of how product features impact retention
   — Two types of feature retention
   — Product feature retention deep dive — Information gain
   — Product feature retention deep dive — MCC coefficient
5. what is the difference in product feature usage between free and paid users? (monetization)
6. what users are doing before using this feature and after it (context)?
7. what is the completion rate of using this feature (simplicity)?

In the next post, I am going to answer the first question from the list above — how popular is a feature among users?