Product Manager Metrics: Process is as Important as Formulas

John Mecke
Jun 28, 2019 · 6 min read

There is a lot of information on good product management metrics, but the process for building an effective program is often overlooked.

Image for post
Image for post
“product” by CreditDebitPro is licensed under CC BY 2.0

Metrics are critical for Product Managers. There is a ton of information on the web about metrics for product managers. Critical Methods every Product Manager Should Track from Product Coalition is a good example. Designing and implementing a Product Management metrics program is a lot harder. We’ll explore the process for developing and implementing a product management metrics program.

Image for post
Image for post
Photo by John Schnobrich on Unsplash

Start With Your Business Plan

A key aspect of OKR is the linkage of corporate, team, and individual goals.
This ensures alignment throughout the organization. Product Managers should strive to identify metrics that align closely with their entire organization.

product manager metrics ltv formula
product manager metrics ltv formula
Above The Crowd

Clarify Calculations & Consensus

A more simplified version is

Image for post
Image for post
DevelopmentCorporate

There are many ways to calculate LTV. If a company is fairly new, it may not yet know the revenue churn percentage of customers. In such cases they need to use a proxy to estimate the typical lifetime duration of a customer. Often average monthly churn rate is used. Established companies usually have definitive data that describes the average lifetime of a customer. Metrics must fit each company’s unique situation.

All parts of the organization that are impacted by a metric must share a common definition of what the metric is, how it will be calculated, and how the results will be interpreted. Marketing, Sales, and Finance may have different views on how LTV could be calculated, but these views have to be reconciled and one definition agreed to. Otherwise the organization whose views were not addressed will ignore or dismiss the metric. The consequences of not taking corrective actions could be serious.

product manager metrics feasibility
product manager metrics feasibility
“Balance scale” by Sepehr Ehsani is licensed under CC BY-NC-ND 2.0

Feasibility & Cost/Benefit

One example are Marketing Influence Metrics. Influence metrics assess how much revenue was influenced by specific marketing campaigns. Usually this involves a manual process of identifying specific sales transactions that were influenced by a campaign. The attribution process is manual and subjective. A fact-based standard does not exist to determine that the campaign influenced specific sales transactions. As a result there is low confidence that the metric is actually valuable and that the time, cost, and effort to produce the metric is worth the benefit it provides. Often these metrics are called “vanity
metrics
” because they make one part of the organization look good, but actually offer no added value to the enterprise as a whole.

product manager metrics commit action
product manager metrics commit action

Commit to Action

One technique that has been used for decades is to establish upper and lower
control limits for specific metrics. Almost all metrics will have some normal
variability, but control limits can be used to determine when a metric exceeds
or falls short of an acceptable level.

control chart
control chart
http://www.syque.com/quality_tools/toolbook/Control/how.htm

W. Edwards Deming popularized the use of Walter Shrewart’s control limits in the 1950s when Total Quality Management became a standard tool in the Japanese manufacturing industry. Shrewart’s control limits represent a three sigma deviation from the mean of samples taken from a process. Processes that are in control have 99.7300% of all the points will fall between the control limits. When performance is outside of the control limits, it is cause to investigate. It is important to investigate performance not only when a metric falls below the lower control limit because of some issue or problem, but also when the upper limit is breached as well. Understanding over performance allows an organization to reinforce and extend the conditions that led to the better than expected results.

All organizations impacted by a metric must commit to take action based on the metric’s results. Control limits provide a convenient and statistically valid
approach to deciding when to take corrective action based on over/underperformance.

Image for post
Image for post

Publish & Share Product Management Metrics

Metrics should be widely available and not restricted to a small select group of people. David Siegel was the CEO of Investopedia for three years. As CEO, he realized that smart decisions are made based on data, and that the greatest
reason for disagreement in organizations is the asymmetrical access to reports. So he instituted three steps to make sure employees had access to data detailing progress across the company: he distributed reports to everyone, held weekly metrics meetings, and let any employee join any email list.

For public companies, there is always a concern about the potential leakage of material non-public information. If an employee cannot be trusted to maintain necessary confidentiality, then there are larger issues than the metrics program that need to be addressed.

retrospective
retrospective
KnowledgeHut

Product Management Metrics Retrospective

Originally published at Development Corporate.

Development Corporate

Musings of a reformed private equity operator on product…

John Mecke

Written by

John has over 20 years of experience in leading product management and corporate development organizations for enterprise firms.

Development Corporate

Musings of a reformed private equity operator on product management and corporate development

John Mecke

Written by

John has over 20 years of experience in leading product management and corporate development organizations for enterprise firms.

Development Corporate

Musings of a reformed private equity operator on product management and corporate development

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store