What is the impact on the retention rate the product features have?

In previous posts, we analyzed product features based on their usage: daily repeatability and daily intensity. Obviously, the higher these metrics are — the deeper users engage with our product.

Today we will continue learning from data and try to figure out how product features could impact retention.

Before we start doing an analysis let’s articulate our assumptions:

• there is a relationship between product features and retention
• different product features could impact retention differently
• probably there could be a small subset of features (core features) that influence retention the most

As soon as we articulated our assumption we are ready to run a retention analysis.

There are several approaches to assessing product features' impact on the retention metric. The most straightforward one is (it’s not the best one, in the next post I will show the better way):

1. calculate [# users] that use a particular product feature in the current period
2. calculate [# returned users] that returned and use this particular product feature in the next period
3. calculate [% returned users] = [# returned users] / [# users]

Let’s start by calculating [# users] and [# returned users] metrics.

precalculated metrics: [# users] and [# returned users]

There are a few important moments that I need to mention:

• there are some features (f.e. feature18, feature37) that have a tiny number of returning users. From my experience, these product features are features that users experience at the beginning of their user journey. They are set-up features.
• in data, we have 4 periods: 2022–19, 2022–20, 2022–21, and 2022–22. But we should use only periods for which we have data in the full next period.

Now let’s calculate retention for each of the product features.

retention by product features.

Let’s compare the figures from the table above to the assumptions that we did at the beginning of this post:

• indeed, there is a strong relationship between features and retention.
• not all product features are equal: some of them have a strong impact, while others have a small impact or don’t have it at all.
• there is a small subset of product features that have the biggest impact on retention.
• there is no constant retention from period to period. As the number of users is increasing from period to period (because of intensive user acquisition efforts), retention is decreasing. That’s why we calculate [Average % returned users] in the last column.

Could we get more insights from the retention table?

I think so.

From my past experience, we can divide the product features into 4 segments:

• core features: retention is in the range (80, 100] percentile
• power features: retention is in the range (50, 80] percentile
• causal features: retention is in the range (5, 50] percentile
• set-up features: retention is in the range [0, 5] percentile

Let’s apply these percentile thresholds:

product features are divided into 4 segments: core, power, casual, set-up

What can we learn from the chart above?

• there are only 8 core product features. Average retention in this segment is from 12.7% to 20.1%. This segment has 2–3x times higher retention than average.
• there are 12 power product features. Average retention in this segment is from 5.7% to 11.4%. Almost all of the product features from this segment have above-average retention.
• there are 14 casual product features. Average retention in this segment is from 0.1% to 5.3%. This segment has below-average retention.
• there are 7 set-up features. The product features in this segment have zero retention, but that’s OK as they need to just set up the product for users and are not supposed to be used further.

There are a few caveats that I would like to mention.

• despite the fact that set-up features have zero retention it does not mean they are unimportant. Some of these product features could have an impact on the overall retention of the product.
• a similar logic can be applied to some other product features. The key here is the retention period which should take into account natural usage patterns. Even if your product usage pattern is weekly, some features could be used on monthly basis or even less often.

As I mentioned in the middle of this post the approach that I described here is not optimal. In the next post, I will show an enhanced version of the current approach.