Product Management & Analytics: what metrics should you be measuring?

Amar Virk
Amar Virk
Dec 14, 2015 · 4 min read

Product organizations increasingly talk about how data is at the heart of their decision making process. But, what does that mean? More specifically, what metrics are they measuring? And what is a good framework to help one think through the entire process?

The objective of this post is to answer these questions and to present a framework that we use when designing products.

Engagement vs Transactional Apps

The first step is to think about what type of application/business you’re building/working in. Most applications can be put into two buckets — engagement apps or transactional apps. Social media networks (e.g. Facebook, Pinterest, etc) are focused on engagement, whereas e-commerce apps (e.g. Amazon, Wish, etc) are focused on transactions.

Image for post
Image for post

At a high level, engagement apps are focused on getting their users to perform actions within the app. For example, Facebook would like you to post on your wall, like other posts and share posts. Spotify, on the other hand, would like you to spend more hours listening to music within the app.

Transactional apps are focused on getting users to complete a transaction. For example, Amazon would like you to buy as many things from them as possible. Uber, would like you to order a car and complete your journey as quickly as possible.

Once you’ve figured out what type of app you’re building, you then need to focus on the Big 4 Metrics.

The Big 4 — Growth, Engagement/Transactions, Retention, Monetization

There are four big “buckets” of metrics:

  1. Growth: Gaining new users
  2. Engagement/Transactions: Increasing usage of the app
  3. Retention: Ensuring that existing users come back
  4. Monetization: Converting app usage to money

Most organizations will focus on daily, weekly, and monthly new user sign-ups as one measure of growth. Beyond that however, the nature of the app often dictates what other metrics to focus on.

Growth can come from:

  1. Virality: Virality is especially powerful in the case of engagement apps where the app can try to get active users to share content or invite friends over an extended period of usage of the app. For example, friends inviting other friends to a Zynga game or to their Instagram network. Metrics: virality factor (k-factor), # of users sharing content, etc.
  2. SEO: SEO works when users and websites create lots of pages that can be indexed by search engines to drive traffic. This tactic can be leveraged both by engagement-driven apps (like LinkedIn, where users create profile pages), and by transactional apps (like Amazon where users upload reviews). Metrics: # of users creating content (profile pages, reviews, etc.).
  3. Paid Marketing: Paid marketing (eg. Google/Facebook ads) tends to work better for transactional apps. These apps, like Expedia or Amazon, usually face fewer steps to monetizing users. This makes the cost per click on each ad affordable. Metrics: advertising cost per click, and transactions per click.
  4. Sales: Sales is usually restricted to business apps that sell volume licenses per deal. Metrics: # of businesses/teams signing up, average team size, etc.
Image for post
Image for post

The metrics worth measuring will vary depending on the type of app (engagement or transactional) you have.

Engagement apps will want to measure some of the primary actions within the app.

Image for post
Image for post

Transactional apps will focus on GMV, total transactions and on optimizing the conversion funnel.

Image for post
Image for post

Most organizations focus on 24hr, 48hr, weekly and monthly user retention. Beyond that, the metrics will vary based on the type of retention method used. Two very common methods are:

  1. Email Notifications: Email notifications have been used for customer retention for a very long time. Metrics: Email open rates, click through rates etc.
  2. Push Notifications: Push notifications are a really effective way to retain customers. According to an Urban Airship study, push notifications are proving to be more twice as effective at retention than email marketing. Metrics: App open rate, session length etc.
Image for post
Image for post

Again, the nature of your app affects how the app monetizes. Engagement driven apps tend to monetize via advertising (eg. Facebook) or subscription fees (eg. Netflix). Transactional apps take a cut of every transaction (eg. eBay, Expedia).

Image for post
Image for post

Summary

Most apps can be classified as either an engagement app or a transactional app and there are 4 Big buckets that are worth analyzing and monitoring. In our experience we’ve found that :

a) Every data-centric startup monitors these metrics on a weekly or bi-weekly basis

b) Every new feature that is being considered should move the needle on the key metrics (engagement or transactional) in A/B tests otherwise the feature may be of questionable value

Product Breakdown

Interesting Product and UX Learnings.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store