Can you call yourself a data driven product manager?

I love working with and hiring product managers that are awesome at product strategy, analytics, design, technology and are great at collaborating across disciplines to GSD.
Those PMs are also called unicorns
If you meet one, hang on to him or her, because awesome all-round athletes in the product world are rare
The more realistic situation is that as a product leader — you construct a team that in aggregate has all the skills you need to build and run a successful product. Most good PMs spike one one or two dimensions (top 5%) , and are proficient on the others (top 25%) — often referred to as the T shaped skill model.
What specific skills do data oriented PMs need to have to be in the top 5% on this dimension?
Business case development:
- This most often happens even before there is an actual product and is needed for executive sign-off or funding. I will write a separate post on what executives look for in a business case before green-lighting a product. On the analysis side however — a PM needs to be able to understand and quantify the value generated by the product.
- Fundamentally — value comes from increasing revenues or decreasing costs. In some cases — estimating the financials is relatively straightforward e.g. A piece of software that is sold for a certain price — often involves estimating total addressable market and market share, or through a more bottoms up estimate using comps.
- But what do you do when the link between the product and core financials is more nebulous? E.g. If the product has been built to increase NPS for the broader organization. PMs often don’t end up building financial models in these situations. However, IMHO — not quantifying the financial benefit of these types of products is almost the easy way out. In the example of the product where NPS is the objective function — there is usually some underlying change in customer behavior e.g. Reduced churn that can be quantified using customer LTVs and experiments
- Tools involved at this stage are typically spreadsheets; ability to read 10ks, or internet research to find “comparable” products is helpful.
Product performance measurement: involves measuring how well your product is doing relative to goals.
- The first part of this is determining how to measure success. I like using the HEART framework for this — which can be used for the whole product, or for new features. Additionally, as I talk about here — equally important is defining the operational metrics to measure quality e.g. response time, uptime, failures rates etc.
- Analytical tools become important here: The product needs to be instrumented with tagging, so that user actions can be tracked. Need to have a tool to aggregate and analyze the data — which can be different depending on the sophistication of person doing the analysis and the analysis itself e.g. something like Mix-Panel for straightforward / standardized analysis on one end of the spectrum; vs. more sophisticated tools like Redshift, Snowflake etc. for more complex / custom analyses
Growth analyses: Debated on whether to include this in the section above, but decided to break this out because of the nature of the analysis here is different. Growth here refers to the act of maximizing your north star metric — and looking at levers across the entire lifecycle. Pirate metrics (AARRR!) is the most commonly used framework here. Major step here is determining the north star metric. This often end up measures such as engagement or retention (MAUs, DAUs), user acquisitions, conversions to some monetization event (e.g. upgrades to premium tiers), number of rides or nights booked. Should be something that measures value delivery to users and aligned with the product vision. Type of analysis often involve:
- Growth Models help you build a model for your NSM, and determine where the leverage is for your product e.g. can you grow MAUs by acquiring 10% more customers or by increasing retention by X%? What is more achievable given your product context?
- Cohort analysis: to see how cohort behavior changes over time, or before and after you make a major product change e.g. new feature or new on-boarding experience
- Funnel analysis: To see which step of the enrollment funnel is causing the most friction resulting in abandons
- Segmentation analysis: to see where the problem spots lie e.g. is a specific market channel not bringing in good quality users, or a specific platform (e.g. desktop vs. mobile app)
- Tools used often end up being the analytical tools described above (tagging, Mixpanel, redshift, decision tree software e.g. CART etc.); building an A/B testing stack in your product is also important to run growth experiments
Predictive modeling — using classical statistics or machine learning. (Broke out classical statistics and machine learning (ML) separately — because ML is increasingly becoming a buzzword, and often a catch-all for all types of analysis that don’t involve a spreadsheet! Very simply — Machine learning is where you let the machine figure out the structure of the model vs. pre-supposing the model structure as in classical statistics. The promise is that algorithms built using machine learning can be more accurate — because you don’t need humans to make assumptions about relationship between variables )
- Building predictions for important user actions like Churn or Conversion e.g. if you know a user is highly likely to churn, you may want to do something in the product to help them realize value; similarly — knowing purchase probabilities are also good for targeted marketing etc.
- Often the constraint here ends up being not building the model — but having an operational platform to score the model in real-time i.e. being able to run a model at a user level, in micro-seconds, while the user is engaging with the product.
- Most PM will need to partner with data science teams for expertise here.
In my experience — good data oriented PMs are expected to very proficient in the first three types of analyses i.e. business case development, product performance measurement and growth analyses. PMs often need to partner with data scientists for predictive modeling — but need to know the potential and applications of these techniques, so that they can use them effectively.
Did I miss something? Drop a note in the comments below
Why you should believe me? I came out of the data / analytics world at Çapital One, and Booz Digital and now lead product, growth and revenue for CreditWise by Capital One.
All views, opinions and statements are my own
