Your Product Metrics Should Include Ratios
The importance of denominators for meaningful data
Let’s jump right into the meat and potatoes — I’ll walk you through a scenario to illustrate one of many reasons why your product metrics should include ratios.
- You are a product manager at Bubbls, a fictional house cleaning service with a subscription model that gives customers 1 or 4 credits a month, depending on their plan.
- Customers can redeem a single credit for an entire house cleaning. Assume all house cleanings are worth the same number of credits, regardless of square footage or complexity.
- If credits are not redeemed by the end of the month, they expire and do not carry over to the next month.
- As the product manager, you are given a list of all the credits in the system. For each credit, you know the date it was purchased and the date it was redeemed.
- You are trying to answer whether there’s an increase in credit utilization in the past few months. How would you do that?
We can begin by creating mock data for a few months and a few users. In my mock data, we are looking at 3 users (001, 002, 003) over a period of 3 months (March, April, May):
Your instinct might be to aggregate the number of credits with a redemption date by month to see if there is an increase in overall credit utilization. When we do that, we can see that there’s a 33% lift in redeemed credits from April to May.
- March: 3 credits redeemed
- April: 3 credits redeemed
- May: 4 credits redeemed
🏆🎉👏 Fantastic, Bubbls must be doing something right…right?
Think again!
What’s the issue? When we are solely looking at totals and not ratios, we aren’t getting the whole picture.
In fact, although it looks like we are seeing a positive trend in credit redemptions, when we factor in the total purchased credits, we see a different story.
While the count of redeemed credits increased over the past few months, the proportion of redeemed credits to purchased credits has decreased. As the Product Manager of Bubbls, this should spark a series of questions such as: Is 4 credits per month too high? Should we have a mid-tier version? Was there a bug or release that prevented customers from redeeming?
Whatever the answers to these questions are, one thing is abundantly clear:
The product metrics you choose to track and look at matter.
As much as we want to look good and feel good, the numbers narrate a story and should drive your decision making. So it’s important to choose the right ones that accurately paint the picture.
Thank you for taking the time to read and for being a part of my product manager growth journey.
Have some thoughts, comments, opinions? Let me know what’s on your mind — after all, “feedback is the breakfast of champions”.
Check out my LinkTree to see what else I’ve been up to. 🚀
