How to Calculate & Benchmark Day 1, Day 7, and Day 30 Retention in Amplitude Analytics
Day 1 Retention, Day 7 Retention, and Day 30 Retention are popular KPIs to evaluate the performance of an app or website product. But how can you calculate them in Amplitude Analytics? And what is a good benchmark to work towards?
What are “Day N Retention” KPIs?
Day N Retention metrics examine what proportion of users return to an app or website after a given time period. Typically people look at:
- Day 1 Retention (“D1R”) — the day after they first use the product (Day 0)
- Day 7 Retention (“D7R”) — the same weekday as first use, but a week later
- Day 30 Retention (“D30R”) — 30 days after first use
The important thing is that all measurements are relative to Day 0 for the user, which is their first day regardless of the calendar day.
As an example, let’s say we have the following activity data for an app:
- Day 0: 100 users install the app
- Day 1: 55 of those users come back and open the app
- Day 2: 40 of those users come back and open the app
- …
- Day 7: 12 of those users come back and open the app
In this case D1R would be 55%, and D7R would be 12%. Notice that we count how many users come back exactly on the day, so the day 2 count isn’t in either calculation. This is technically known as bounded retention, and can be contrasted with unbounded retention, which would include unique users coming back on the day or afterwards. Generally bounded retention is preferred, because unbounded can constantly change as new days of data arrive.
Once again, it doesn’t matter which calendar day the users installed the app — those 100 installs could all have been on February 14th, or 10 per day from February 1st to February 10th. Either way, all the user activity is aggregated and summed up, relative to their first day.
Alright, that’s the maths done! Let’s look at how we get these numbers for your product in Amplitude.
How to Calculate Retention KPIs in Amplitude
It’s just two clicks to create your first Day N Retention chart.
You’re done!
The Y-axis is showing Day N Retention, where N is defined by the X-axis value. In this example, the yellow circle is D1R (75%), the purple is D7R, and the orange is D30R. The blue line is called a retention curve.
The retention value is aggregated for all users who had their first use of the product during the time period of the date picker (by default last 45 days).
Typical things you will see:
- The chart starts at 100% on day 0 — by definition, unless you have some custom settings, all users are active the day they first use the product.
- The chart slopes smoothly downwards to the right, and hopefully flattens out (more on that later).
- There might be little spikes — these could be caused by automations you’ve set up, like an email or push notification that goes out to all users on their 5th day.
Be Aware of the Details
Starting and Return Events
In the top left of the chart, you’ll see dropdowns for “Starting Event” and “Return Event”.
By default, Amplitude chooses “New User” for Starting Event i.e. the first time Amplitude gets data for a user (based on device or session ).
Similarly, Amplitude defaults to “Any Active Event” for the Return Event. Events can (and should) be categorised as Active or Inactive in the Govern section of Amplitude, where Inactive events are those the user doesn’t do themselves. For example:
- Push Notification Sent — this is done by the server, the user may never see it, so this should be Inactive event.
- Push Notification Opened — this is done by the user, probably the first thing they do in that session, so is an Active event.
If you have Inactive events set as Active, your user numbers and retention metrics will be inflated and easily gamed (just send a notification to everyone every day!).
Typical variations for the dropdowns are to change the Starting Event to Sign Up or Purchase, and the Return Event to a valuable action (e.g. send a message for a chat product). I recommend you experiment with changing the Return Event for the sake of comparison — if a lot your users come back but don’t do anything valuable, there is little purpose in retaining them and likely they will churn eventually.
Segment Filters
By default Amplitude will include all users, but you might want to focus on a particular segment e.g. those from a particular Country. You use the segment pane in the top-right as usual to do this.
Similarly, you can group by Country. This allows you to compare Retention curves against each other and spot which perform better (the “higher up” curves).
Daily, Weekly, or Monthly
We’ve mainly be thinking about Daily retention, but if it makes more sense for the natural usage frequency of your product, then you can flick this setting to Weekly or Monthly.
The principles for Daily/Weekly/Monthly are exactly the same, just the unit of time period on the X-axis has changed. You’ll probably also find it harder to find good benchmarks because many products are focussing on daily engagement!
The Change Over Time Chart
A common technique is to make a cohort analysis of retention. That lets us answer the question, “Is Retention improving over time for each new set of users that discover the product?”.
People typically use this approach to examine the impact of changes they make to their product. Don’t overlook marketing changes though — if higher quality users are acquired, you will also see stronger retention.
You can jump to this view by choosing “Change Over Time” in the middle panel of the retention chart.
You’ll see a chart something like this:
What you’re seeing here is the unaggregated version of the previous chart. This means we split out each date of the starting event on the X-axis, and mark their Day N retention with coloured lines.
For example, the blue line is Day 1 Retention, and the highlighted point is Day 1 Retention for users who were new on April 20th. So we can say that 83% of the users that were new that day came back on April 21st.
What you’ll be hoping, is that all the lines are gradually going upwards, indicating new cohorts perform better than older ones, and therefore that your product or user quality is improving.
It’s not unusual to see cyclic patterns in this chart — different weekdays often have different associated quality of acquired users, or different likelihood of engagement. For example, SaaS apps might be more likely to be used on a work day, so D1R will be strong for a Monday when the next day is Tuesday, but weak on a Friday which is followed by the weekend.
Evaluating & Benchmarking Day N Retention
When building a product, you’re probably aiming to achieve and increase your product-market fit.
Product-market fit can be seen to have a very distinct shape on a retention curve, and then is that it converges or levels off.
If your curve levels off at 10% rather than 5%, it means you’re retaining on average twice as many of the users you acquired.
Often this will be the same users coming back every day, but sometimes it might be a larger group of users coming back every few days. Either way, you have a loyal segment of users.
Also of interest is how quickly your curve levels off — this might hint at the effect of your activation strategy, product triggers, or something else!
Benchmarks
Andrew Chen mentions that D1R of 60%, D7R of 30% and D30R of 15% is very good.
Also often referenced is this set of curves from Quettra for DNR of the top Android apps.
I would suggest you first focus on getting your curve to level off at some level, and then after that try and hit the level of these benchmarks.
How to Improve your Retention Metrics
You would probably love that your analytics data could also tell you exactly how to improve your DNR metrics, but unfortunately that’s a lot harder than simply measuring them!
That said, I would recommend a few initial steps that might give you ideas for areas in which to improve:
- Slice and dice your retention chart — by country, acquisition channel, and any other property that feels key to your market. If you see groups that have stronger retention, ask yourself if there’s a way you can get more users like that.
- Look at short term and long term retention. Efforts to improve short term retention (e.g. D1R) are often easier to action, and can lead to some (often less) improvement in long term retention.
- If you have the Amplitude scholarship plan or a paid plan, try out the compass chart, and look for user behaviours which correlate with retention. Ask yourself if those behaviours could also be causal, and if you could find a way to encourage more users to do the same things.
Hire a Retention Expert
There are so many more ways to examine retention in Amplitude and generate insights about your product.
I work as an analytics consultant specializing in Amplitude, and help growth stage startups dive deeper into topics like this.
If you want to hire someone who has worked on improving retention in many different contexts, get in touch.
