Measuring user adoption
This is part 3 in a series of articles about measuring Key Experience Indicators (KEIs). In this series I go deeper into the Google HEART framework for large-scale data analysis. The framework was put in place to help choose and define appropriate metrics that reflect both the quality of user experience and the goals of your product. Each article in the series discusses one of the HEART dimensions — Happiness, Engagement, Adoption, Retention, and Task success. Enjoy and use it!
What is adoption?
In the context of products and services, adoption is the act of beginning to use something new.
Considering new features and new users, there are four types of user adoption (see my user adoption model below):
- Internal adoption: When existing users begin using new features. For example, the percentage of existing Instagram users who adopt a new story feature within 1, 7, or 30 days of its introduction.
- External adoption: When new users begin using existing features. For example, the mean number of days new Instagram users create their first story from when they opened their account.
- Adoption flags: When new users adopt new features. A green flag is raised if they’re successful, and no red flags are raised when they’re not.
- Routine adoption: Happens when existing users adopt existing features.
User adoption is an unbiased behavioral measurement and is therefore trustworthy, valid, and reliable.
Why measure adoption?
Understanding people’s adoption behavior toward a thing (feature, service, process, etc.) is extremely helpful in identifying whether or not the thing is providing value. When people are quick to try something out for the first time, it means they care about the problem it is set to solve and that they have high expectations of it. High adoption numbers mean that your thing has promise.
Key mistakes in measuring adoption
Only measuring overall adoption. While overall product adoption is an extremely useful metric to track, it is a business rather than experience metric. When you only measure overall product adoption, you are not going to learn anything about adoption with the usage of the product. For example, “Our overall product adoption rate is 88%” means that you have a lot of new users. It may be useful for business stakeholders (in Marketing, Sales, etc). “The adoption rate for the new sharing feature by existing users is 2%” is an extremely actionable metric. You may conclude the feature is not very valuable or not well promoted or unusable. A very quick follow up research study can reveal the reason and help devise an action plan.
Confusing adoption with engagement. Engagement measurements reveal how involved people are with the product, how much they use it. Adoption only focuses on new usage, either internal (new features used for the first time by existing users) or external (new users beginning to use existing features). I’ve seen people use adoption and engagement interchangeably and confuse between them. While they are related, they are not the exact same thing. Nit picky, yet using terminology consistently and correctly increases comprehension and shared team understanding.
Three adoption metrics
Adoption rate: The percentage of new users of a feature. The formula for calculating adoption rate is: Adoption rate = number of new users / total number of users. For example, if you have a total of 1,000 users, of which 250 are new, then your adoption rate is 25% (250/1,000). The adoption rate should always be calculated for a specific time period. For example, if you calculate an adoption rate for the month of July, you would use the total number of users who used the feature for the first time any day between July 1 and 31. You will then divide that number by the total number of users on July 31, the last day of the month.
Time-to-first [key action]: The mean time it takes a new user to try an existing feature, or an existing user to try a new feature for the first time. That time can be associated with understanding the value of the feature, getting curious about it because of its name and promise, or context that makes the feature an attraction. For example:
- Time to first click a navigation item from when a user opened the homepage is 4.7 seconds.
- Time to first usage of a hotel concierge service from checking-in time is 16.5 hours.
- Time to first transaction on an eCommerce website from when an account is first created is 21 days.
I would recommend on identifying key actions with the product or service first and not measure this metric for every single small action that can be taken.
Percentage of users who [performed key action] for the first time: A slightly different way to examine the first time experience. What percentage of users have performed an action you care about for the first time in a given time period. For example, “86% of users purchased at least three products through our mobile app in the month of July.”
Taking action on adoption data
User adoption data is actionable. Here are some examples for actions you can take with it:
- Start an onboarding team and set adoption goals. Challenge an onboarding (or any other) team to get better scores for user adoption by setting goals. Set adoption goals for critical, revenue-generating features and for internal and external adoption.
- Look into perceived value. In case your adoption rates are not high or as high as expected, conduct qualitative research to better understand what makes new users adopt existing features, and existing users to adopt new features.
- Conduct first click tests. First click tests are helpful in identifying what are people’s first choice for clicking given a specific task. Combine it with a short conversation and you are on the path to improve adoption rates.
You only have one chance to make a great first impression. Adoption metrics provide you with solid data and wisdom about that first impression from the perspective of your audience. Adoption metrics are behavioral, unbiased, and actionable. They are extremely helpful for large-scale data analysis and uncover an important aspect of the user experience.