Product people KPIs aren’t about the product
While product people are worried about the ‘one metric’ for their product or velocities for their engineering teams, the product community doesn’t talk much about the KPIs for product people.
There are many Medium and Quora posts that discuss the best KPIs for products. When we ask about KPIs for product people specifically, we get a conflation with the product itself. Or worse: a focus on missed deadlines and budget issues.
All of these are wrong. Let’s fix that.
‘Good’ product people and Venn diagrams
Before we dive into how we observe a product person is good (or great), we need to define what we think makes product people should be doing.
Mind the Product and Product Manager HQ have fairly good definitions of what product people are. They even include the obligatory Venn diagram of what product people focus on:
My short summary would be that product people should be good at building team alignment towards a strategy or vision. They develop a product context through talking with customers and stakeholders. They make sure the team is always working on what is most important.
‘Good’ KPIs
Lean Analytics has a definition I like to reference about what makes ‘good’ metrics:
- Not so confusing that you have to explain it
- Comparative (e.g. new vs. churned users)
- A ratio or rate (e.g. % monthly active users)
- Helps you take action or generates change, not just for vanity
- Auditable
KPIs are a small number (<5) of ‘good’ metrics that are meaningful to the success of the group.
Teams succeed together
A misconception is that product success is directly tied to the actions of individual product people. This just isn’t true. Product success is based on the work of the whole team (and their managers and stakeholders).
Deadlines and other budgeting isn’t directly in the hands of the product person either. It is dependent on many factors: engineering experience, engineering skill, unknown-ness of the projects, complexity of the requirements, known-ness of tools being used, how beta the technology stack is, quality level assumed, acceptance processes, and others. Product can have a hand in these processes, but the whole team either meets a deadline or doesn’t.
Finally, having the team achieve together against goals builds team cohesion and camaraderie. A good example is leaderboards. Stacking individuals has a negative effect on motivation while working as a team against other teams improves performance. See section 28.2.2.1 of “How to Avoid the Dark Side of Gamification: Ten Business Scenarios and Their Unintended Consequences” or “Points, Badges, and Leaderboards: The Gamification Fallacy.”
As an aside, I see product people in certain orgs getting most of the credit or taking most of the blame for how well a product does. It is surprising to me that it continues this way when a lot of people will profess they don’t know what product managers do… maybe we all need scapegoats?
Qualitative or quantitative product people?
Now that we see the difference between product’s and product people’s success, how do we decide what to measure about the product people themselves? Are there ‘numbers’ that can be created that track this at all?
Unfortunately, the soft skills that make product person good (e.g. communication, alignment, strategy, etc.) are difficult to collect automatically as metrics. We can ask our teammates directly, using qualitative research methods, to understand how we are doing as product people. Just like we would with our customers about their problems we are hoping to solve.
There are guidelines on what I would consider ‘good’ qualitative research:
- Observe people, in their own environments
- Don’t ask them leading, biased, or yes/no questions
- Do ask them open ended questions, what they have done in the past
- Synthesize in chunks to see patterns
The questions you should be asking your team
You should be talking with your team to understand how well you are doing, just like you would with your customers.
First, you should already be doing retros to understand how the team is working together. At Philosophie we try to have retros every sprint (weekly), but will sometimes do it every other sprint (biweekly). But this is still about team performance, not about the product person necessarily.
Second, you should be asking questions about your own performance on some regular cadence as well, but individually, so that the impact of alignment is judged where it matters. Due to the time sink of doing this with all team members it seems like every few weeks is probably good enough in some rotation.
These are the questions that I have found to be most valuable:
- What is the strategy for the product, as you see it, and how do you feel about it?
- What do you need to do right now for the product?
- How do you feel your work contributes towards the high level goals and strategy of the product?
- How do you feel overall about the work you are doing on the product?
If you are have one-on-one meetings with individuals every few weeks these questions shouldn’t take more than 15 minutes per session to answer and the ROI is huge.
While it is great to understand individual issues with your performance as a product person, it is most interesting to identify patterns that come up over and over. This is a great way to understand what you need to do to grow as a product person.
Getting better at feedback
Recently, our team has been focusing on feedback. Both for employee growth and directly between team members.
A teammate at Philosophie, Adam Thomas, recently recommended “Thanks for the Feedback.” I realized a lot of things about how I accepted feedback and how much I could improve.
It is a book filled with helpful mindsets and methods for teasing out helpful information from any feedback you can get. Even when it doesn’t seem helpful on its face or outright hostile. I would recommend reading it for anyone that works in teams.
Along with the questions above we need to understand the feedback in a way that is actionable for us. This is why in-person one-on-one meetings are most effective rather than surveys or other anonymized methods. We get to ask questions and follow up on things that we don’t (yet) understand.
Ask them regularly
As product people, we already have the tools to understand our teams and improve our role in them. Asking these questions regularly is the first step towards building truly great teams.
In the end, product people shouldn’t be judged on the success or failure of their product, but rather on how well the team is aligned and understand how what they are working on is important.
