Input-Output Metric Framework for Product Managers
Problems that a Product Manager Faces in defining metrics
- Unable to get your products prioritized because the company’s priority is not the metric you are moving to.
- You have chosen the wrong metric for your product and figure this out later after launching the product.
- I have not been able to conclusively achieve the success of the products you have launched.
- I have been dependent on other teams to achieve your success metric and have been unable to do so.
- Have achieved success metric for your own product but later you figured out that you have screwed some other metric for some other team in the process of doing so.
The success of a Product depends on choosing the right metric
The difference between a successful product and an unsuccessful product is the selection of the right metric to measure the success of the product. Without a measurable success metric your product is directionless and hence there is no way to figure out when to take what actions. It’s like a race when one doesn’t know where the finish line is.
What is the product metric?
These are well defined quantifiable and directly measurable entities that determine the progress made by a product and provide input to further improve the product to achieve better results.
Why are they important?
Because :
- Metrics define a product decision
- Helps in effective communication and collaboration with stakeholders about what is important
- Helps to identify problems and take a corrective course
- They help to provide a feedback loop for the product launched.
- Ensures that product success is identified and recognized throughout the organization
What are some product decisions dependent on metrics?
- Estimation of the potential impact of the product
- Prioritizing amongst conflicting initiatives by normalizing products to a common metric
- Scoping user stories within an initiative based on the highest impact first
- Defining the features of the product based on the metrics to move
What are the different types of products?
There are two different types of products on which metrics can be defined and they are as follows :
- New Product Inception Metrics
- Optimizations/Additions on top of existing product metrics
Piece of Advice: If you don’t have the right measurable metric, you should not build the product, and conversely if you chose to build a product ensure you have the metrics in place to measure success.
Ask the right questions to get your metric?
Primary Output Metric
Question — When will you call your product a success or a failure?
Why is this metric important?
This is the most important measurable outcome of your product which establishes the true impact of your product. The reason why this is important :
- This is the most important key result for the product and significantly helps to align stakeholders to start product development.
- Over a period of time product managers should be able to judge the success and failure of the product based on this metric and also figure out if additional product investments are required
- Before building the product PM’s should estimate the change in the success metric so that they can set their targets accordingly
- This metric should be defined in such a way that it is measurable and can be tracked on a regular basis. (if possible should be traceable without an experiment)
Eg: Monthly ad revenue (See Case Study Below for Details)
Input metrics
Question — What will be the important milestones your product will reach before it becomes an overall success?
Why is this metric important?
These metrics help to segregate the monolithic success metric into various independent secondary success metrics which are functions of different components of the product. The reason for the importance of this metric is as follows :
- Assign the success metric for each of the stakeholders to ensure proper accountability
- Use these metrics as milestones to ensure that the primary output metric is right on track.
- Break the monolithic output metric equation as a function of dependent input variables to achieve the output metric
Eg: Click Through Rate * Cost Per Click, Fill Rate of Ads (See Case Study Below for Details)
Check metric
Question — Are you screwing up some other metric of the company while building your product?
Why is this metric important?
These metrics prevent the moving round and round in a circle paradox where the net value the product delivers to the company is zero. The reason why this is important :
- It helps product managers to think more holistically about the general good of the company rather than only their own stream.
- Also since most metrics are interconnected and improvement in one metric may lead to the degradation of the other and hence this fact should be known while making product decisions.
Eg: Orders/discovery session on Swiggy (See Case Study Below for Details)
External Dependent Metric
Question — What is the most important external metric which the product does not control and which is required for your product to attain the desired primary output metric?
Why is this metric important?
The success of the product is also dependent on the right inputs and if these right inputs are not achieved the desired outcomes are not reached. This metric can be of two different types :
- Type 1: The input metric is not dependent on your product and is a function of the market forces which is beyond your control.
- Type 2: The input metric is a dependent metric and is either the outcome metric of another product.
Eg: Total monthly impressions on ads (See Case Study Below for Details)
Best Practices of defining metrics
Few Practices to define metrics :
- Make your primary output much independent from other teams as much as possible. (For Eg: at Zomato if a product manager is responsible for bringing discounts from merchants than his primary output metric should be y% of GMV funded as discounts by merchants and not x million $ of discounts)
- Always define the increment in the metric that your product will bring. (For example: Increase metric y from x1 to x2 and not Increase metric y to x2)
- Always define a time period as to when that metric will be achieved. (For example: Increase metric y from z1 to z2 within x months)
- The input metrics can also be attributed to different teams/stakeholders so that there is clear accountability on the product.
Iterative Product Development
This is a simple approach to product development based on metrics. The key points here are as follows :
- Check for external dependent metrics and ensure that they are right or normalize input/output metrics so that the output metrics become independent.
- Iterate the product to improve the input metrics.
- Once input metrics are right after a lag period check the output metrics and they should 99% be right. (If it does not happen you have defined the wrong input metrics)
- Once primary metrics are achieved, then see the check metric to confirm that your product is not screwing up the other metrics.
- If you achieve your output metric within the time period then your product is a success.
Case Study: Let’s apply the framework in three different problem statements
Case 1: Amazon Prime Subscription
Case 2: Netflix Recommendations
Case 3: Uber User-Based Pricing
Change in metrics defines the roadmap
The way product features are added to a product is changed by the metrics in which that particular product is chasing and these metrics for a product change with time leading to different features on the product roadmap to optimize for the newer metrics.
Let me know your thoughts and improvements in this framework in the comments and do use this framework in your products.
