Which Product Metrics to track?
This is Part 8 of a long-form series of setting up Product Event Analytics correctly.
You could read Part 7 to understand the decision of the Nomenclature of Events and Parameters for your Product.
If you already understand the Product Metrics you need to track for your Products, feel free to skip over to the next section where I discuss how to conduct the analysis, draw insights, and build your own Product Dashboard.
There’s a ton of material out on the Internet that would teach you which Product Metrics to track for your Product. You can check out these popular guides on figuring out Product Metrics in the links below —
If you want to understand this topic deeper, check out this book.
Let’s start with the basics of Product Metrics and how to derive the right metrics based on your use case.
Product Metrics are a representation of your Product that help you find out whether your Product, its features, and modules are actually achieving the Business Objective for which they exist.
Since all Products exist to achieve a business objective, first let’s focus on this business objective with which Products are launched.
Business Objective
Most Products are created to fulfill a customer’s need while fulfilling a Business’ Objective of either earning revenue or building more efficiencies to reduce expenses.
Products could lead to the fulfillment of business objectives directly or indirectly.
Let’s take a few examples to understand this on a deeper level —
- Fintech Application for Investment in Financial Securities — In this case, the Business Objective would be to get users to invest in financial securities since such a platform would charge a commission on each investment made by users from an Asset Management Company.
- Cab Booking Application for booking shared rides Online — In this case, the Business Objective would be to get users to take cab rides after booking online since such a platform would charge a percentage commission on every completed ride from a Driver.
- Driver Application for responding to shared ride booking Online — In this case, the Business Objective is still to get users to continue to complete cab rides after booking online.
- Social Media Application for sharing personal updates Online — In this case, the Business Objective would be to sell as much inventory of ‘Advertising slots’ to advertisers. To fulfill this, the potential advertisers would need users to show their ads to. Hence, such a business would have its objective of getting as many users to use their application as possible.
Now that we’ve understood a bit about ‘Business Objectives’, let’s understand how to tie them to a ‘Product Objective’ or a ‘North Star Metric’.
North Star Metric
The famed ‘North Star Metric’ has been talked about a lot already.
A North Star Metric is the commonly aligned goal amongst the Business and Product teams that the Product Application needs to target, increase or optimize, and which should be tracked with a mutually decided upon frequency such as ‘Daily’, ‘Weekly’, or ‘Monthly’.
The frequency of this North Star should depend upon the frequency with which users use a Product.
Borrowing from the Business Objective, let’s discuss the Product objectives as follows —
- Fintech Application for Investment in Financial Securities — In this case, the Product Objective would be ‘Average Number of Securities Investments per user’. It’s important to note that the number of securities invested per user could be ‘weekly’ or ‘monthly’ depending upon the volume of the active user base.
- Cab Booking Application for booking shared rides Online — In this case, the Product Objective would be ‘Average Number of Rides Booked per user’. It’s important to note that the number of securities invested per user could be ‘daily’ or ‘weekly’ depending upon the volume of the active user base.
- Driver Application for responding to shared ride booking Online — In this case, the Product Objective would be ‘Average Number of Rides Accepted per Driver’. Cab Booking Application for booking shared rides Online — In this case, the Product Objective would be ‘Average Number of Rides Booked per user’. It’s important to note that the number of securities invested per user could be ‘Daily’ or ‘Weekly’ depending upon the volume of active user base booking a ride.
- Social Media Application for sharing personal updates Online — In this case, the Product Objective would be active usage which could be defined as the ‘Average Number of Posts per user’. However, in this case, there could be more than just one definition of ‘Active Usage’. The frequency of tracking this ‘Active Usage’ would be on a ‘Daily’ or ‘Weekly’ basis.
Level 1 Metrics
The North Star Metric should be broken down into the supporting level 1 metrics that lead to an increase in or optimization of the identified North Star Metric. The way to come up with the level 1 metric is to think of the ‘phases’ of a Product and then come up with a key metric to track for each phase.
Let’s take the same example of an online ride-booking application. Here are the typical phases to think about —
Acquisition
Acquisition is defined as when the user comes up and registers on your ecosystem of products.
Acquisition is a key phase to solve for that would increase the probability of improvement of your North Start Metric by getting more users.
Some of the most common Acquisition measures are as follows —
- App Installs
- Signups
Mostly, Acquisition is measured by App Installs if you’re an App only product or by ‘Signups’ in the journey after App Installs.
Your Product in the ‘0–1’ or in the ‘1–10’ phase must have Acquisiton as the key level 1 metric to track and monitor.
Your Product must have a healthy mix of Signups from ‘Performance/Paid Marketing’, ‘Direct’, ‘Organic’, and ‘Referrals’ for this to be sustainable. Normally, ‘Performance Marketing’ shouldn’t be greater than 25% of your acquisition sources mix.
Activation
Activation is defined as the value moment of your Product. Post experiencing this value moment, a user is considered as having a high chance of becoming a repeat user.
Similar to ‘Acquisition’, ‘Activation’ is a key phase to solve for that would increase the probability of improvement of your North Start Metric by getting more users to try your service.
Some of the most common Activation measures are as follows —
- Activation Rate
- Time to Activate
For our example, an Activation moment would be when a user completes his/her first ride.
The target for activation would be to get as ‘Percentage of users completing signup to activation’ to keep on increasing, most commonly defined as the Activation Rate.
We’re targeting the percentage of users from signups since the total count may increase if the absolute number of signups increases by that much.
Mostly, activation must be measured in percentages. For our case, we should measure the ‘Monthly Activation Rate’.
Please note that even though ‘Acquisition’ may not be a primary target for a Product in the ‘0–1’ and ‘1–10' phases, ‘Activation’ would continue to be a target of importance in all phases of your Product journey.
Engagement
Engagement for a Product is the measure of how well the users use your Product for its desired objective.
‘Engagement’ is a key phase in your Product when it succeeds to gain the initial traction that you want. Engagement on your platform would ensure repeated usage and hence would increase the probability of improvement of your North Start Metric by getting more users to continue to be with you.
For most products, I’ve seen ‘Active Usage’ being touted as a measure of Engagement. While this is a great measure of Engagement, ‘Active Usage’ has to be defined in the context of your product. Normally, it should be the repeated activity on your Product that’s meaningful for your users to derive value or for your Product to have the probability of monetization.
Some of the most common measures of Engagement are as follows —
- Active Usage
- Stickiness
For our example, moments of Engagement or ‘Active Usage’ of our product would be when the user searches for cabs for a destination.
The target for engagement would be to get as ‘Percentage of users who open the app to people who actively search for a cab’ to keep on increasing.
We’d still target the percentage of users since the absolute count can keep on increasing with increasing signups or installs.
Let’s also briefly discuss a much-touted Engagement metric called stickiness.
Measure for stickiness is monitored once your user volume and usage frequency crosses a threshold. Hence, in this case, until we reach a stage of Daily Active Usage being significant, it’d not be insightful to measure stickiness.
Please note that even though ‘Engagement’ may not be a primary target for a Product in the ‘0–1’ phase, but would be a primary target in the ‘1–10’ as well as the ‘10–100’ phase.
Retention
Retention for a Product is a measure of how deep the users perceive the value of a Product to be to continue using it.
Some of the most common measures of Retention are as follows —
- N-Day Retention
- Unbounded Retention
Whatever form of retention you aim to measure, it’s important to define the ‘Event’ for which this retention is being measured. As discussed earlier, the event of importance would differ by the type of Product ranging from ‘Active Usage’ for a Social Media Platform to ‘Orders’ for an E-Commerce Platform.
For our example, the ‘Event’ of importance should be ‘Bookings’.
The measurement of Retention can be divided into two phases —
- 0–1 Phase — In the initial phase, it’d be insightful to measure Unbounded Retention since that would help inform how many users continue to book over a period of time.
Hence, Unbounded Retention in this case would be defined as the (Number of users who booked another ride by the end of a quarter/Number of users who booked a ride at the start of a quarter) - 1–100 — In the post Product Market Fit phase, it’d be insightful to measure N-Day Bounded Retention with a 30-Day Retention, to begin with, and to move to a 7-Day Retention as the frequency of usage or booking goes up per user.
30-Day Retention would be defined as the Percentage of users who book a ride again within a month of the total users who booked a ride at the start of the month.
Please note that without retention, your Product would always face a leaky bucket problem which would state that without a stable retention rate, most of the users you acquire, would continue to churn. In the case of a Digital + Service Product Platform such as E-Commerce purchase and delivery or online cab booking, retention of a Product becomes a function of the experience of the service as well as the Product. Hence, in order to tackle this problem, you’d need the support of your Program team as well.
Monetization
Monetization metrics are a measure of how much revenue you’re able to generate from your users.
In a D2C context, ost Monetization efforts concern themselves with increasing this Monetization per user.
Therefore, the right measure of Monetization in our example could be the Average Revenue Per User. However, without the presence of additional Value Added Services to be provided to a user, it’d not be meaningful to measure Average Revenue Per User since alone, it’d just be a function of pricing.
Level 2 Metrics
Having covered all the traditional phases of a Product, let’s take a deeper look at the Level 2 Metrics which would contribute to the increase or optimization of the Level 1 Metrics discussed above and hence, what to construct on your dashboard for Product to continuously monitor —
Acquisition
For Acquisition, you should be measuring the following metrics of importance —
- Signup Breakdown by Sources
- App Install to Signup Completion Funnel Percentage For App
- Home Page to Signup Completion Funnel Percentage for Website
Activation
For Activation, you should be measuring the following metrics of importance —
- Signup to Ride Completion Funnel Completion Percentage
Engagement
For Activation, you should be measuring the following metrics of importance —
- App Open to Cab Search Funnel Completion Percentage
Summary
In all, the above metrics chart is only a starting point and you’d quickly notice that your Product Metrics of importance would keep on evolving and changing the more your users continue to use your product and derive some sort of value from it.
Every Feature or Module of work that you envision for your Product, should affect a Level 1 Metric directly and should have its own success measurement in terms of ‘Funnel Conversions’, ‘Flow Reports’, and ‘Retention Reports’.
Once you build a ladder of Metrics that is important for you to track, you should build a ‘Feature’ or ‘Module’ wise dashboard that should ladder up to your ‘Level 1’ and ‘North Start Metric’ which should be included in one dashboard.
In the next sections, we would look at how to construct a Product Analytics dashboard using one of the tools discussed to actually see this in implementation.
Do share your feedback with me at kshitj.saxena@gmail.com or connect with me on LinkedIn
