How Should You Measure a Product Manager?

Rich Mironov at Product Bytes weighs in with a thoughtful post about measuring product managers. He had a recent experience in Sweden where the product executives he worked with shared the same concerns as product executives in the U.S., Europe and the rest of the World, namely: what is the right way to measure the performance of a product manager?

This is a really interesting and difficult question to answer. Rich breaks it down into three areas: product-level metrics (product revenue, profit, customer sat), team-level metrics (are the other teams getting what they need from product management), and individual-level metrics. The first two metrics are fairly well defined — there is a strong history of measuring products and teams with these criteria and there are MBA programs with entire curriculum defined about measuring product performance. Unfortunately measuring individual product manager performance is more tricky as these metrics are not as clear.

Lacking traditional or clear metrics, most companies rely on product-level metrics such as product revenue, margin, or customer satisfaction to measure the performance of an individual product manager. The thought process is defensible — you want the product manager to be motivated by metrics that lead the product toward success. Unfortunately, product managers typically have very little control over the outcome of these metrics, or at best, indirect control. For a product manager measured on product revenue, the product manager could spend months designing the perfect product, training the sales channel, and working with marketing to design a lead generation campaign — only to wake up one day and find that the executive team had made a strategic decision to make a large acquisition and change Sales resourcing. That change would have a huge impact on the product manager’s metrics, at no fault of the product manager; effectively they would be penalized for doing the right things. For a product manager measured on product margin, I always ask “Do you have the ability to outsource the development of your product to a 3rd party if your internal engineering team delivered an estimate that was too expensive?” For most product managers, the answer is a resounding “no.” Customer satisfaction suffers from a similar control issue — there are lots of reasons beyond the product that customer sat may suffer, and most of those reasons are not something a product manager can impact.

The other issue with measuring product managers on product-level metrics is an issue of timeliness. If you are measuring your team on product revenue, you can make an argument that it may take up to 18–24 months to know if the product manager is doing a good job. Consider this — if you hire a new product manager, before he or she can impact product revenue, the following will need to occur:

• The new product manager will need to go into the market and research the market’s problems (up to 3 months)
• The product manager will then write what they have learned into a business plan, and get it approved (1–2 months)
• Then the product manager will write requirements for engineering (up to 3 months)
• Then engineering will build something (6–9 months)
• Then in most products there will be a Sales cycle (3–6 months)

Only then will we have the data to understand if the decisions made on the front-end of the process are “good.” Hopefully we will be more agile and most faster, but the point remains — using product-level metrics to measure individual performance is insufficient. We need a better way, and there is a better way.

What I have settled on for measuring individual performance is to measure the activities required for a product manager to be market-driven:

1. Setting a quota for market visits (10/quarter is a good start IMO),
2. Being able to defend an updated business plan in front of me and their peers every quarter,without using the phrases “I think,” or “In my opinion” (which requires higher order thought and research).
3. Keeping their finger on the pulse of the business by defining and measuring the right product level metrics and communicating them in the form of a dashboard report monthly.

I like these metrics because they require a product manager to get out from behind their desk and be outside-in focused, and the product manager can control them, as opposed to revenue, margin or CSAT which the product manager cannot control.

All things being equal, it would be preferable to measure outcomes as opposed to activities, but it is wicked hard to separate and quantify the inputs of a product manager vis-a-vi all the other variables that go into making a product fly. Activities are a sufficient proxy to measure a market-driven product manager to supplement product-level metrics and understand if a product manager is doing their job — without having to wait two years to find out.

If you like what you’ve just read, please hit the recommend button below!