Picking a metric — More outcomes, less outputs please

At the end of each quarter, OKR and KPI driven companies are busy determining best metrics to measure success for next quarter. This triggers a slew of debates and analysis to align on a few to focus on. Every time, I find myself having this discussion in one way or another to trim the list of output metrics so we can focus on the outcome ones that indicate delivering users value. Hence, I thought I would document it.

Before I dive into suggesting metrics

Let’s take one-step back and talk about two types of product teams.

• Type 1: Teams focused on outcomes and celebrate when they have helped the user get the job done
• Type 2: Teams focused on features in a roadmap of ideas and celebrate when they have shipped something

I hope we are aligned that type 1 teams will be more successful at reaching their goals compared to type 2 teams who depend on their best hunch features. Else, check out the article here or here making a strong case why one is better than the other.

What are the metrics?

If we are aligned that outcome-focused team are preferred over output-focused teams, then let’s prefer outcome metrics instead of output feature metrics. Let’s take an example. If our goal is to have the most engaged users,

• Output feature metrics are number of users who used feature X, Y, Z (registered, saved search, gave 5 star review, …)
• Outcome metrics are number of users that return within 7 days, average sessions a week, monthly active users

The outcome metrics are indications that the product added value to the user and hence the user comes back beyond a single transaction (and hopefully more often).

The output metrics are just guesses that the user reached value if they used feature X and probably prefer the product to get what they want.

I’m not arguing that teams shouldn’t focus on driving feature adoption. Each team should have their best guess at which feature(s) would lead to the desired outcome and move the outcome metric for the product. If at the end of the day your guess is wrong that a feature leads to an outcome, then there is no point in further driving that output feature metric. You are more likely to catch this if your team’s eyes are on the outcome, instead of the output.

Aside: Most companies todays are aiming to form a long-term relationship with the customer, then let’s aiming for a long-term metric of return usage to indicate engagement instead of the transactional metric of having used a feature or not.

Need more metric ideas?

Apptentive has a great cheat sheet of metrics depending on where in the funnel you are currently focusing on. It’s a great starting point to brainstorm and adapt to your needs.

But it’s hard

A pushback that I often get is that moving the outcome metrics (such as daily active users or monthly active users) is hard and we have not succeeded in the past. But isn’t that the point of why we need to focus on outcome metrics even more?

If we are doing a lot of work but not moving the needle, then that’s the best indication that we are focusing on doing the wrong things and should find the right thing before we run out of runway. At the end of the day, avoiding the problem will not solve the problem.

Here’s a rally to go out there and find ways to add more value (& validate it by looking at outcome metrics) instead of creating more feature clutter (& making ourselves feel better by looking at output metrics).