Image credit: Venkat Komoju

Internal scorecard of a Product

A product’s success hinges on both, the happy engineers who write code, and customers whose one or more pain points the product addresses.

A product goes through it’s own life-cycle, from conceptualization to delivery to usage, with the product building process consisting of various teams lead by a product manager.

External scorecard and customer satisfaction score calculation have become common, especially amongst SaaS vendors, whereas, there are very few techniques of arriving at an internal scorecard of a product.

Internal scorecard

The internal scorecard of a product is defined as the average of various NPS style scores collected from functional units that touch a product, viz. Sales, Support, Customer Success, Product Management, QA and Development. The scores can be collected monthly or quarterly.

Here’s a table analyzing an aggregate survey response after a release cycle:

The arithmetic mean of the individual scores is the internal scorecard of the product, which comes to 71 for the above sample set.

A visual representation of the scorecard on a radar chart could be plotted after every data collection exercise to examine the hotspots, and could also tell a better story. For example, a point in time plot from data in the above table (in blue as Q2), clearly shows that Product Management is super happy with what they are building but Sales ain’t so much! If you compare that with the earlier Q1 data (in orange), it shows that this is a new problem to tackle.

Internal vs. external, why measure the internal?

Most organizations happily measure the external scorecards, but external success may hide a carpet bombed engineering organization. Quick fixes, tightly coupled integrations, and forked code lines lead to long term technical debt but a short-term win in the market. A product manager’s job is to ensure that a product not only wins the market, but wins the heart of an engineer who toils everyday.

Bottomline: Are you winning as a Product Manager?

How do you know that you are winning as a Product Manager? When your score on the internal scorecard is more than the one on the external scorecard, the ratio of which can be summarized as:

Internal score
-------------- >= 1
External score

The PM now knows that there are internal fences to be mend, when the above falls below 1.

Summary

In order to succeed as a Product Manager, a balance of happy engineers and customers is needed. A data-driven approach for achieving this balance is an NPS like survey for the product’s internal stakeholders.

Caveats

1. The responses from the functional department could be misleading, may require confrontation/corroboration with hard data viz. revenue growth, number of support calls, renewal and churn, bug count vis-a-vis past velocity. As an aside, there is no deterministic way to measure technical debt.
2. You may need to develop weights for each department to arrive at a weighted average rather than straight-forward arithmetic mean.