Two types of feature retention
In a previous post, I showed you how to analyze a feature's impact on retention. My initial definition of feature retention was like this:
[# returned users ftr] =
- [# users] that used a particular product feature in the current period
- and then returned and used this particular product feature in the next period.
The definition above can be used when we want to figure out which features are habitual. The logic is the following: if the user uses this product feature and it delivers value to him, he returns and reuses it. This forms a habit.
This approach is reasonable, but it will not work for all product features in your product.
Let’s consider a few cases.
In your product, there is a content subscription. After the user subscribed he will get notifications as soon as new content is available. Despite the fact, that the user returns to the product using the notification feature, it’s clear that the notification feature does not deliver real value to the user. It just unlocks it for him. The real value the user will get when he consumes new content.
Another case is set-up features: they are used at the beginning of product usage to set up the product and will not be used anymore after that. But these features can be critical to make sure that future user experience will be as nice as possible.
That’s why we could create the 2nd definition of feature retention:
[# returned users prd] =
- [# users] that used a particular product feature in the current period
- and then returned and used any product feature in the next period.
The new definition is focused on retention in general. It does not require returning and reusing a particular feature.
Let’s calculate [% returned users ftr] (definition from the previous post) and compare it with [% returned users prd] (definition from this one).
As we can see results are somewhat different:
- The metrics [% returned users ftr] and [% returned users prd] have a different range of values.
- In the Top 10 lists, only a few features coincide.
I think it would be useful to compare them via scatter plots as well.
By adding the 50/80 percentile lines to the chart we can easily grasp insights about which quadrants are empty and also what the density of product feature points in each quadrant is.
Also, I colored different product feature points with color gradients to remind us that points with light colors have much fewer users inside.
In the left scatter plot, features like feature4, feature18 and a few more have very small [% returned users ftr]. Their values are in the range from 0.1% to 0.6%. It means that these product features are mostly used just once.
Nevertheless, users who used these features have [% returned users prd] around 7.5%. It means that these product features did not break the user experience and users continued using the product. So there are no issues with them, they are just specific.
A question that we could ask ourselves is: whether there is some relationship between these types of retention.
Let’s build another scatter plot and check it (I filtered out product features that are used by less than 2% of users):
As we can see there is a positive relationship: as [% returned users ftr] is growing, [% returned users prd] is growing as well.
It means that the better a specific product feature will cope with local retention, the higher the global retention we could expect.
Having 2 definitions of retention which one is better to use?
There is no definitive answer here.
I would say that both of them are correct and give us a different perspective on how product features impact user behavior.
To summarize:
- high [% returned users ftr] signals us which product features are useful and clean. It’s a tactical view and it could be useful to a Product Manager that is responsible for a specific product feature or a subset of a product.
- [% returned users prd] signals us which product features impact overall user experience. It’s a strategic view and it could be useful to the Head of the Product.
