Definitions of Success
On The importance of KPIs
One of the critical skills that a product manager must possess is the ability to define what the success of their product will look like. As the ultimate owner of a product, PMs are responsible for ensuring the success of their product and also communicating what that success (or lack thereof) will look like to the organization as whole.
In order to do this, PMs must define certain metrics prior to launching a product or feature, that they will monitor and analyze over the lifetime of the product. These metrics are critical for a few reasons, namely:
- They are the metrics which PMs will rigorously optimize for in future product enhancements and iterations
- As they are used to judge the ultimate success of any product or feature, it is on this basis that future funding and resource allocation may be given, or taken away.
Not only do PMs have to define the metrics, but they also have to project the actual numbers that they would like to hit with every metric they track. This is a somewhat nebulous exercise so in order to do so effectively, PMs must posses a deep knowledge and understanding of their space and competitive landscape, a clear view of differentiated features in their product, and past experience that could be pertinent to the upcoming product or feature launch.
The metrics that I have used to define success with my products are called KPIs or Key Performance Indicators. When I define KPI’s and forecast my numbers, I like to keep three principles in mind that help shape effective metrics.
KPIs must be Simple
Software products are complicated and the number of interactions your users can have with your product are vast. As a result, the metrics that you could hypothetically track are also vast and could become quite complicated.
When defining KPIs however, I like to think about the singular, core problem that my feature or product is trying to solve for users, and zero in on simple metrics the help me understand whether or not we are solving that problem.
I launched an application called Rhapsody SongMatch early last year. It allowed people to identify any song currently playing out loud much like Shazam. It also had discovery features that would allow users to explore similar artists, similar tracks, and editorial posts for matched songs and artists.
While I was interested in how much users were interacting with the discovery features, the core problem we were trying to solve was to help users identify the one song that was currently playing. As a result, I focused my KPIs around the singular interaction of matching a track. A few specific KPIs were match attempts per user and successful match rate.
KPIs must be Measurable.
While it may be nice to have qualitative KPIs, it is much more important that your KPIs be quantifiable, and thus definitely measurable. Qualitative data is open to interpretation and opinion, quantitative data on the other hand is not.
Why is this important? Because there are always going to be other products or initiatives competing for the same resources that you need. If your KPIs are qualitative, other people may be able to cast doubt on your KPIs and in turn, drive resources away from you. Fortunately, people cannot question cold hard data.
What does a quantitative KPI vs a qualitative KPI look like? Qualitative would be, our app will be the best app in our category. Well what exactly does “best app” mean? Everyone on the team will have a different definition. If you break this KPI down into quantifiable metrics however, there is no room for opinion or interpretation.
If you want to measure whether or not your app is best in category, your KPIs should be downloads per day, star rating, new 5 star ratings per day, new 1 star ratings per day etc. Each of these metrics can be tracked and they can also be tracked to a certain extent for your competition. With these metrics, you would be able to accurately asses, with data, whether or not your application truly is the best in your category.
KPIs must be Realistic
This does not actually refer to the metric you define, but to the number your forecast for the specific metric. And I’m not to saying that you shouldn’t be bold with your KPIs. You absolutely should have ambitious plans for the success of your product, but it must be grounded in reason.
If you were going to launch a new search engine that would compete head to head with Google, a KPI could be “market share of search traffic in North America.” When you set your number for this KPI however, an unrealistic forecast would be 50% within the first 6 months after launch. Lets be serious, no matter how amazing your search product is, there is not a shot in hell you are going to change the habits of 50% of the american people within 6 months.
A more attainable measure would be “percentage month over month growth in search traffic.” With a KPI like this, you could target a 10% to 15% month over month growth in search traffic, which would be a great growth rate and is a completely realistic measure of success.