Measuring user adoption

Tomer Sharon
Jul 30, 2018 · 5 min read

This is part 3 in a series of articles about measuring Key Experience Indicators (KEIs). In this series I go deeper into the Google HEART framework for large-scale data analysis. The framework was put in place to help choose and define appropriate metrics that reflect both the quality of user experience and the goals of your product. Each article in the series discusses one of the HEART dimensions — Happiness, Engagement, Adoption, Retention, and Task success. Enjoy and use it!

What is adoption?

Considering new features and new users, there are four types of user adoption (see my user adoption model below):

  1. Internal adoption: When existing users begin using new features. For example, the percentage of existing Instagram users who adopt a new story feature within 1, 7, or 30 days of its introduction.
  2. External adoption: When new users begin using existing features. For example, the mean number of days new Instagram users create their first story from when they opened their account.
  3. Adoption flags: When new users adopt new features. A green flag is raised if they’re successful, and no red flags are raised when they’re not.
  4. Routine adoption: Happens when existing users adopt existing features.

User adoption is an unbiased behavioral measurement and is therefore trustworthy, valid, and reliable.

Why measure adoption?

Key mistakes in measuring adoption

Confusing adoption with engagement. Engagement measurements reveal how involved people are with the product, how much they use it. Adoption only focuses on new usage, either internal (new features used for the first time by existing users) or external (new users beginning to use existing features). I’ve seen people use adoption and engagement interchangeably and confuse between them. While they are related, they are not the exact same thing. Nit picky, yet using terminology consistently and correctly increases comprehension and shared team understanding.

Three adoption metrics

Time-to-first [key action]: The mean time it takes a new user to try an existing feature, or an existing user to try a new feature for the first time. That time can be associated with understanding the value of the feature, getting curious about it because of its name and promise, or context that makes the feature an attraction. For example:

  • Time to first click a navigation item from when a user opened the homepage is 4.7 seconds.
  • Time to first usage of a hotel concierge service from checking-in time is 16.5 hours.
  • Time to first transaction on an eCommerce website from when an account is first created is 21 days.

I would recommend on identifying key actions with the product or service first and not measure this metric for every single small action that can be taken.

Imaginary example for a potential adoption metric to be measured on the Apple website: Mean time to first click on the Fridge item.

Percentage of users who [performed key action] for the first time: A slightly different way to examine the first time experience. What percentage of users have performed an action you care about for the first time in a given time period. For example, “86% of users purchased at least three products through our mobile app in the month of July.”

Taking action on adoption data

  1. Start an onboarding team and set adoption goals. Challenge an onboarding (or any other) team to get better scores for user adoption by setting goals. Set adoption goals for critical, revenue-generating features and for internal and external adoption.
  2. Look into perceived value. In case your adoption rates are not high or as high as expected, conduct qualitative research to better understand what makes new users adopt existing features, and existing users to adopt new features.
  3. Conduct first click tests. First click tests are helpful in identifying what are people’s first choice for clicking given a specific task. Combine it with a short conversation and you are on the path to improve adoption rates.

You only have one chance to make a great first impression. Adoption metrics provide you with solid data and wisdom about that first impression from the perspective of your audience. Adoption metrics are behavioral, unbiased, and actionable. They are extremely helpful for large-scale data analysis and uncover an important aspect of the user experience.

Other articles in this series

Tomer Sharon

Written by

Head of User Research & Metrics at Goldman Sachs, Author of Validating Product Ideas and It's Our Research, Ex-Google, Ex-WeWork, WWE fanboy. 2∞&→