How to Systematically Improve your User Retention
Over the last 5 years I helped Pinterest grow from 20M MAUs to over 200M MAUs, leading the growth engineering team and leading Pinterest’s user retention efforts. Most recently, I’ve also been helping early to late stage startups understand and achieve long-term sustainable growth. Through these experiences, I’ve found that the single most important and often overlooked driver of long-term sustainable user growth is healthy user retention. Most companies don’t see the critical role retention plays in building a long-term business, and if they do they find it difficult to actually improve.
In this post I’ll be sharing why retention is so important and some key tools that are instrumental in improving user retention and directly applicable to companies at any stage.
Why is retention so important to your business?
Whether you’re an early-stage startup starting to think about scaling user growth or later-stage with hundreds of millions of users like Pinterest, user retention is really the silent killer, as outlined in this great post by Brian Balfour. It could hinder your product from truly taking off early on or severely limit your product’s long-term success. Twitter is an example of the latter, as their user growth has effectively plateaued even after registering the bulk of the internet population. This is largely due to poor user retention.
User retention is also core to a successful business, here’s how the typical business model looks like for most consumer or enterprise companies:
Users are acquired through various means either paid or organic, of which a percentage of those users will retain long-term, and eventually go on to generate revenue for the business. As you can see, user retention sits at the heart of the business, where increasing retention can effectively lower the cost to acquire a monetizable user, increase lifetime value, and ultimately increase overall revenue for the business. You can imagine what it means for a business if your product’s user retention was low or trended to zero… gg.
So, given its importance to your business, how do you improve your product’s user retention?
Understand your user retention
The first thing you need to do is define what a retained user means to your business. Like any north star metric, this should be specific to your product and be a practical measure of a user realizing enough value from your product to use it on a regular frequency. For Pinterest, it made practical sense for users to visit us weekly to discover new inspiration for their lives, so we chose weekly active usage (WAU) as a measure of retention. If you’re a chat app like Whatsapp, long-term retention would more suitably be daily active usage. Figure out what makes the most sense for your business.
Next, you’ll want to understand what percentage of your new users retain weeks and months after registration and how that changes over time. A simple way to visualize this is by constructing a new user retention graph like the one below. There are a number of tools that can do this for you off the shelf such as Mixpanel, but it’s also fairly easy to implement yourself.
This new user retention graph is valuable in understanding how your new users retain, the health of your retention and where to focus your attention to improve retention. Some key insights you can grok from this visualization are:
- Indication of a long-term retentive product. If your retention curve ultimately flattens out like in the graph above, this is a strong indicator that you have a long-term retentive product. However, if it trends towards zero or even if it flattens out fairly close to zero, you’ll need to improve your core product ASAP. You can’t build a sustainable business without a healthy percentage of your users retaining long-term, since eventually you’ll run out of users using your service.
- Changes in retention over time. In the graph above, it shows that the newer cohort in blue is retaining worse than the earlier cohort in red. You should be understanding how your retention changes over time and be concerned if fewer users retain over time. This is another way your user retention could trend to zero. Many times this could happen due to a new mix of users you’re acquiring, in that case you should be evolving your new user experience accordingly.
- Your window of opportunity. The graph above shows that 50% of new users don’t return after the first week. This is a strong indicator to focus even more on improving your initial onboarding and first-use of your product. Of course every product will retain differently, but it’s very common to see that the window to retain a user degrades over time, which is why it’s so important to deliver core product value as soon as possible.
Every product’s retention will be different, so by charting it out you’ll be able to understand it and identify where to focus your retention efforts.
Learn from your retained users
Now that you have an understanding of how your product retains and the window of opportunity, next you’ll want to figure out how to actually improve your retention. There are a number of general best practices to consider, such as: making your initial experience super simple, understanding and delivering upon a user’s intent, forming habits early on, etc. Beyond that, the primary thing I recommend is learning from your already retained users. Assuming you have a retentive product, you’ll have some set of users that took the right steps to realize that magic moment with your product, which you can then use as a blueprint to further improve your product.
Talk to your users
The easiest way to start, especially early on when you don’t have a lot of data, is to talk to your users and learn what worked and didn’t work for them. I suggest talking to your retained users to start since they’ll be most willing to share their experiences. User research is important in understanding exactly what users did to ultimately see value in your product as well as the struggles they faced. Incorporating a process around regularly talking to your users and marrying it with user engagement data will provide valuable insights on what to improve at any stage of your product.
Analyze your retained users’ journey through data
As you acquire more user engagement data, you can look historically to identify which users retain and the actions they took through your product. Using this data you can optimize your product to guide even more users down these magic moments to improve retention.
- Identify a set of core actions, specifically actions that would indicate that a user understands and gets value out of the product, these correlate well to long-term retention. For Pinterest, this could be viewing pages of Pins, clicking into Pins, saving Pins, creating Boards, etc. I recommend starting simple with a few key actions.
- For a cohort of users (e.g new users over a given week) calculate the percentage of users that engage with each action. Think of this as a step in a user’s journey towards retention. (i.e saved 0 Pins, saved 1 Pin, saved > 1 Pin, etc)
- For each of these steps, compute the percentage of users that retain. Here’s a graph of what that might look like for a single action, where the bubble sizes represents the percentage of users reaching each step.
4. Use this data to prioritize feature development and validate causality through experiments. In the graph above, you can see that there’s a large opportunity in getting a user to save at least one Pin, based on the percentage of users and correlation to improved retention. Some might call this an Aha Moment where a user realizes core value, represented by the step-change in long-term retention. Using this insight, you can prioritize experiments to validate that getting users to their first Save does in fact improve long-term retention.
This process gets even more interesting as you segment this analysis by different user cohorts and include more actions. This is obviously a simple model and you can take it further by optimizing the selection of actions accounting for confounding variables, as well as optimizing the discovery of key user actions rather than doing it manually. However, a simple model like this can identify a plethora of opportunities to optimize your product and move the needle for retention.
Rapidly learn, iterate and improve
Now that you understand your user retention and what you could do to improve it, the next important step is being able to run as many experiments to rapidly iterate towards improving your retention. What you’ll quickly find with your experiments to improve retention is that they’ll need to run for weeks if not months to observe statistically significant improvements to long-term retention, making iteration impractical. To address this, invest in building good leading indicators of long-term retention which you’ll use instead to predict improvements to retention. This will help you make quicker decisions on experiments and increase overall iteration throughput.
- Identify a set of leading indicators, specifically actions early in a user’s journey indicating that a user retained value out of the product. For example, say you’re trying to improve the first session experience, consider a metric like performed [action] in next session rather than performed [action] in first session. It’s still an early enough action and it better indicates that a product improvement led to retained value.
- For each potential leading indicator, take a cohort of users and compute:
- Coverage, the percentage of users that perform the action.
- Precision, the percentage of users performing the action that actually retain.
- Recall, the percentage of retained users that actually performed the action.
You should get something like this:
3. Select the leading indicator that has high coverage and a healthy balance of precision and recall. In the example above, Indicator #2 is the best candidate because:
- It has high coverage which allows you to obtain more users in your experiments, which is crucial to enabling faster ship or shutdown decisions. Indicator #1 is the most accurate leading indicator, but it will take 7x longer to accrue the same number of users in a given experiment.
- It has a relatively high average of precision and recall (i.e F1 Score).
- Although the F1 Score is the same as Indicator #3, it has higher precision which is more important for a leading indicator since you’ll be optimizing your product to get more users to perform the action.
Leading indicators aren’t perfect and there are cases where you’ll want to run your experiments for a longer period of time just in case, but they are a critical tool to increase the velocity at which you learn and iterate to improve retention, while trading off some accuracy.
In Closing
Healthy user retention is critical to the success of your business but it requires the right understanding, tools and rapid iteration to actually improve it.
- Understand how your users retain on your product by defining and charting user retention over time.
- Figure out how to improve your retention by learning from your users through user research and behavioral data.
- Rapidly experiment, learn and iterate by investing in leading indicators that predict long-term retention.
In my experience, these three tools have been immensely valuable in significantly improving user retention and should help you obtain a better handle on this key problem while laying the foundation for building a long-term sustainable business.
Follow me if you liked what you read and would like to learn more about user growth. In the next post I’ll be diving into 3 Key Tips to Retain Your New Users. Stay tuned!
Special thanks to Casey Winters, Mo Shahangian and Scott Tong for reading early versions of this post.
