Touch wood!… no no, PMs always say Touch Metrics! Keep measuring…
An assumption can only let you make more assumptions… A hypothesis can only let you initiate or begin with… Only right amount of data and analyzing it right at right time, will let you make great decisions…
There are three different types of businesses
- Any moment a CEO or the Director wandering got an idea (a great product or a great feature). Assuming everything just like, Sunday is always a best day to holiday, absolute in love with the idea the next minute. The next moment calls the poor product manager, sells his/her dream, gives the plan, asks to get the roadmap ready and start implementing.
- Any moment a CEO or the Director reading the industry trends or from any piece of information received from internet (there is deep shit on internet), finds something like ‘industry will be x bn by 2030 or a competition launched a nuclear test’, builds his/her hypothesis and explains same to product manager to start working on it.
- Any moment a CEO or the Director gets an idea from any source (deep shit internet, his/her own mind, customers, industry network, competition, etc.), first thing he/she does is to meet with relevant people to discuss the next steps, where few or more experiments (call it [Market] research) involved to gather as much as information and evidences of the idea or feature to make it a success.
Well it might not be only CEO or Director (of Product/-). Don’t be offended please. It can be any individual in organization. I used these two roles above as they hold the highest authority on the product and business around the product. Hence it is highly important that, in an organization, at least they are in the third type.
And what’s the outcome? You might have read — 90% of startups fail or 99% of business die in a year or xx % of startups die before launch, etc. Whoever efforts and finds this data and however genuine these kind of information is, what’s true is the results mostly are from the first two types of businesses.
We talked above about when the product or idea is getting started. There is other one too —
- One while initiating a product/idea/feature which we discussed three types above
- Second after launching the product
A product goes through several phases in its lifecycle from ideation till maintenance or death. And all through its journey there is one constant process
Keep collecting enough data and measuring
Measuring what?
- Are we going in right direction?
- Is it performing right as expected?
- How my customers understand my product?
- How much satisfied my customers are?
- Are my customers referring me to others in their network?
- Am I making targeted revenue?
- Am I making any profit?
- Am I making unit profit?
- and many more… (truly based on your product/service/feature)
And who has to do this, keep collecting data and measuring? — The Product Manager.
A product manager should know what to measure, how to measure, and when to measure. If something is not measurable then DON’T DO IT.
So, you are a Product Manager and if your boss (from first two types above) just wakes up and gives you a new idea or feature — first take them to third type. Give your plan to build enough evidences to help him/her decide whether to continue with the new idea or sleep over it.
First Part; When an idea or new feature comes in mind
There are few questions to answer. But, the first question you should answer is:
WHY??????
You should answer why should we work on new idea or build new feature? Are these the kind of answers for it —
- it will make our business grow
- out of x customers, y have asked for it
- our competition launched same and made x % growth
- this feature will be our USP
NO…
The answers to the WHY should be out of —
- Whether the new idea or feature is aligning to your vision & mission.
- Whether the new idea or feature is helping in achieving Objective(s) set for the current FY.
- Whether the new idea or feature is helping in attaining one of the key results identified for the current FY.
- Whether building this will add any value to customer. Whether any pain points getting solved for a customer.
And before saying ‘Yes’ to all four above questions, knowing HOW these above four points are happening to be the answers for WHY is very important.
You have answered the WHY and the decision is to ‘continue’, then the second and next questions to answer are
- Which all or how many customers showed interest for buy in.
- Who customer is ready to pilot/beta (if possible).
- Is anticipated go-to-market time still relevant. Maybe the idea is only needed now (one time or periodically once in some period/a year) for a set of customers.
- What is the pricing strategy that’s acceptable or will fly in market.
- Is competitor having similar or better solution for the problem? How they have solved it.
- What is the competitor performance on the feature/idea if they have already solved it.
- And many more i.e. relevant or needed for you to answer whether you should continue building the idea/feature.
All these above questions can be answered only by doing:
- Market Research
- Competitor Analysis
You may not be able to answers all the questions but you will definitely gather enough data/metrics that can help you measure and decide whether to proceed with feature/idea or not.
Also to keep this article short, Market Research and Competitor Analysis are whole different topics which I won’t be able to cover here. Those need separate articles or watch more for my upcoming book where I will be explaining how to do them in detail. You can also very well learn and practice surfing about them online. There are lots of materials already available.
So, to continue, if you read my approach above to decide on the idea or a new feature,
- was there any assumptions?
- were there any hypothesis?
No right. We don’t take the decision. We have to derive to the decision. Then only we can assure on it.
Second Part; When that new idea or feature is built and launched
You did the great research. You analyzed everything and answered the WHY. You got the development started, built the product/feature and launched it. Now what?
- Market well and let’s see how many customers buy it.
- Showcase how many customers are using it, no matter how many of them are activated or paid.
- Let’s get the next enhancements ready as per roadmap.
Right? No… Not right…
Firstly, on this idea or feature, there is a budget that got spent/burnt on answering WHY’s and knowing HOW’s and doing WHAT’s.
Secondly, there are no signs that whether the next set of enhancements required.
Then, how to continue?
Metrics… Get them first.
Yes the metrics to measure. I said in the beginning, you being a product manager have to keep collecting data and keep measuring. Now the product or feature is in market and customers can use it, you have to work on collecting the data to measure its success and measure whether next set of enhancements required (if so then what is required).
What to measure?
Well, any data you collect to measure, all the metrics fall under any of these two category
- The Usage — Defines Health. Like engagement, clicks, activities, repeat usages, usage patterns, user journey trends, etc.
- The Performance — Defines Growth. Like activation rate, retention rate, Cost-to-Benefit ratio, increasing Customer Lifetime Value, decreasing CAC, etc.
You can very well use popular frameworks like AAARRR, HEART or GAME. All are very profound frameworks and will help you measure the success of product or feature. I would use another space to explain these and of course covered in my to be soon coming book.
But for now, I would like to simplify bit more for you, and the reason is, I just don’t want you to follow one, rather learn the basics of how to identify and design your metrics.
I have experienced this, where a PM created beautiful dashboard having all the metrics — DAUs, MAUs, AOV, LTV, AUV, Churn rate, and what not. Looking at dashboard was a good feel but I was looking for a data that can tell me whether to kill the feature x as I haven’t seen healthy usage of it and wanted to cut cost. The metrics drawn and results showcased to me was nowhere helping me to decide whether to kill the feature or maintain it or let the enhancements go.
Maybe it is quiet difficult to draw what to measure and how to measure. Let me talk a simple and easy way to do so.
You don’t need all the metrics in the world to know something about your product or a feature. There is only one key indicator that will give you the information. First thing to do is to find that one key indicator or what in our product terms called as “North Star Metric”.
Start with your North Start Metric. Find it.
Well, what makes a north star metric?
This is the single metric; the numbers, which will tell you whether you are growing or dying.
So, for any product you should be figuring out, what is that if we don’t do/optimize, we will die.
Hey that’s Revenue Praveen. All businesses have same North Star Metric then? No, sorry. Revenue is not a one death key indicator for any business in this world. Of course, revenue is everything that has to be grown every year. But to grow your revenue and travel from -ve or zero-to- +ve unit economy, you will have to optimize SOMETHING. Yes, that something is your North Star Metric.
What could be that indicator?
- Think about your product and tell what that most significant value your product adds to your customers.
- Think about your company vision and mission. And tell how with your product serving to customers, the company is able to reach its vision.
Both above will narrow to one answer. That value is your North Star Metric. The value you have added to your customer due to which you are making business is your North Star Metric.
E.g. Facebook North Star Metric is Daily Active Users. How? What value do Facebook add to its customers — you can be connected with your friends, family, and relatives. You can build a community of like-minded like you for a cause or any goodness. Overall you use Facebook because it brings you together and keep connected with people who matter you most. Let’s say the North Star Metric is common one — revenue and Facebook revenue is from ads. Facebook to optimize its North Star would run more and more ads which will lead to decrease in user engagement and eventually decrease in ad impressions and CTRs too.
Instead the North Star Metric being DAU, Facebook teams would focus and work towards optimizing active usage of platform per user. The entire strategy is different from above being focused on getting more and more ads to focusing on increasing engagement rate by providing user specific feed. More the active users, more Facebook can run ads and hence grow.
North Star Metric is one which considers the value addition for all the characters (I don’t know what to call it) involved. What I try to convey here is, take e.g. of YouTube, where you have 3 characters— the content creators, the viewers/watchers and the business (YouTube itself). Maybe I am complicating. Simply put, this metric makes all the user segments and the business happy. Even in Facebook e.g. the post creator, the advertiser, the viewer, and the business (Facebook) itself. The DAU metric makes all of them happy. Post creator reaching maximum views and getting more likes is happy, the viewer getting the content suggested right for him and his positively experienced active time on Facebook made him happy, the advertiser able to reach his targeted audience and get average impressions and CTRs is happy and lastly Facebook getting paid by ads making revenue growth is happy.
Ok, now you got your North Star Metric, will that be sufficient? Of course to the max reach it is. But, there are cases where you will be in need for Supporting Metrics
Supporting Metrics
Why it is called Supporting Metrics? Because it is related metric to North Star Metric and is not independent or drives decisions independently.
E.g. Facebook North Star DAUs are growing. You are told to run more ads and grow revenue. But are you sure you can run more ads? Or what is the healthy ratio where the DAUs doesn’t drop and you make good revenue by running ads. Maybe you run ad after every 4 feed i.e. in 4:1 ratio. Don’t assume, just build the supporting metrics where the user engagement rate is not dropped and you can run ad. This intersection and measuring the health of DAUs and MRR is what your supporting metrics would be. The supporting metrics will always help you to know when to re-strategize on your North Star Metric. Say 4:1 ratio might not remain same for long and DAUs are dropping again. You would get back to your strategy board again and come up with next strategy.
Great, after your North Star Metric, develop your supporting metric(s). There might be zero or more. Just don’t harden yourself to find one at least.
So, you got your supporting metrics too. Your dashboard is almost ready. Do we need more? Yes, the Key Lifters.
Key Lifters
Or Up-lifters? Don’t know what to call them. Maybe you can call them supporting metrics itself.
Optimizing North Star, DAUs will help Facebook grow but, when thinking of optimizing, there are n different features like, feeds, events, posts, marketplace, etc. which internally run as sub-product. You as a Product Manager would need the key indicators showing significant growth in ‘Usage’ and ‘Performance’ of these features. What if feature Ad Manager usage has dropped by x%. What if feeds are growing irrelevant to the user. Should we call feature x usage decline as its retirement and bring it down. Why feature y usage has sky rocketed by 70% which is not normal. Why 28% of our customers converted only after adding x feature as free in package. How do you measure all these? Well, by using the feature’s usage and performance metrics — the key lifters.
You can see the impact of these feature’s usage and performance is directly affecting the Product and business growth, resulting in increase/decrease of North Star Numbers.
Why I call them Key lifters because they uplift North Star Metric, directly or indirectly.
Too long read isn’t it? Ok, let‘s quickly summarize before closing.
Product Manager should keep collecting data and keep measuring…
There are two phases in Product where you approach differently to measure.
- When you got or found an idea or a feature
- When product or feature is built and launched
When you got an idea or feature, first thing to do is, answer the question WHY? Then, do the Market Research and Competitor analysis to answer other questions. The approach is to decide whether to build the idea/feature OR just sleep over it.
When product or feature is built and launched, you as a product manager should measure two things —
- the usage
- and the performance
To do so, follow the approach,
- Define/Find/Draw your North Star Metric and strategize to optimize it.
- Derive/find Supporting Metric(s) to help you measure the heath of your North Star Metric
- Define Key lifters (Up-lifters) to help you measure unit wise which feature/sub-product usage & performance is uplifting/down shifting North Star Metric.
Keep Measuring...
Still feel this is complex and stuck with it? You called my approach complex then. Thanks for everything.
Well, I can still help bit more… take this easier approach…
Do every needful to figure out whether your customer is able to get/find what your product is intended/built for.
You are a booking.com then, find whether and how many customers are able to find a hotel on your site and book one.
You are a marriage/dating app, find whether and how many customers are able to find love/life partner on your platform.
You are a YouTube, find whether and how many customers are able to search what they are looking for and find it in first page of search results.
This is so powerful and so much connecting approach that you don’t need all the metrics in the world and just keep optimizing that one thing what your customer is coming to you for.
I like to say again… Keep Measuring…
Signing Off…
Thank you!
