Measuring what’s important right now: Lead-lagging metrics

TL;DR: Most product initiatives benefit from using two metrics to measure their success, and to many product managers’ surprise, they are not always outcomes.

While it is always worth striving for measuring outcomes in product management, it is actually not always the most important thing to measure. I’ve heard countless discussions around “Is this an outcome?”, instead of the more qualified discussion: Is this important to measure? In this article, we will explore how to determine and measure what is truly important based on the current stage of your efforts.

Outcomes

According to the Cambridge Dictionary outcomes are the result or effect of an action, situation, or event. In product management, we can define it as:

The results a feature is intended to achieve for users, the customer, or our business.

In a perfect world, we would be able to connect things we do — our bets — to a business outcome. Mainly because the main job of a product manager is to maximize the success of a company through its product. The problem is that sometimes the distance between the business outcome — like purchase/growth — to an individual feature you are building is very far. I’ve earlier described how to use models to solve this problem. But you also need different measurement approaches at various stages.

Lead-Lagging Example: The Cake

Imagine that you own a café and would like to sell more. You consider getting a new coffee brand, improving the indoor atmosphere, training the waiters to upsell, or baking a delightful cake. After analyzing each bet, you decide that the delightful cake has the best value/effort. You could then create a model for this bet:

Find ingredients → Mix and Bake Cake ↔ Test with Friends (iterative until delightful) → Delightful Cake → Present cake in cafe → Sell Cake

Considering this model, we can now examine what to measure:

The cake: Lead-lagging metrics

In my experience, it is most constructive to have two metrics. In all stages, metrics can have different significance, and a lagging metric is almost always more significant, but they are in nature not very helpful to measure success early on. There you need a leading metric. So we use two metrics. Here is what it could look like for our cake bet in a model:

Measuring what’s important

There can be many factors that affect the final business outcome. So it is about measuring at the right level.

Lead-lagging metrics

The leading metric

The leading metric should focus on what you are currently doing. This should:

• Measure the closest meaningful metric to your activity to see results

The Lagging metric:

The lagging metric should measure something that is important, and measures the nearest end goal:

• The nearest important outcome

The nearest outcome is a user, customer, or business result. So it is something that happens because someone does something. In the cake example, the final result is for the café to sell more. But that’s not what you should measure first. It would be more productive to measure feedback from users (which is a more uncertain outcome, because your friend might lie to make you feel good — and so does other users), but it is important to have an idea of what “delightful” is.

Different stages: The Cake (Continued)

Some typical measurements in different stages are:

Let’s take three parts of the cake bet:

Early stage

While the end goal is to sell more in the café, having this as a main focus while baking the cake is not really helpful because it gives us no insight into if we are moving in the right direction. We would like to measure “success” early on. Therefore we set up lead-lagging metrics:

• Leading: In an early stage, is it definitely meaningful to talk about having all the ingredients to bake the cake. So progress is valuable to understand. While we could have an outcome, it will be more lagging, and we want to have an idea of success early on.
• Lagging: We could make the initial lagging metric “sell more”, but the further away you get from the end goal, the less meaningful it will seem. Your only focus can’t be selling when you are trying to create a delightful cake. The meaningful outcome is that the cake is actually delightful — here rated by a friend.

It is not the same thing that is important at different stages. In the early stages, it’s often measuring something we can learn from fast. To learn fast, we need to rank our metrics in order of learning speed:

The easiest thing to measure is if we like the cake ourselves. But this might be highly biased. Inviting friends to sample requires a little more, inviting a customer even more. So in this case, we find that friends' feedback is the right balance for the nearest important outcome.

Middle stage

At a later stage, when the cake is present in the cafe, you really want to measure if the cake is being sold. At this stage, the lag metric is easy to measure: From the counter, you can measure if the cake is being sold. But you still want to learn: Here waiters could ask for feedback about the cake’s delightfulness.

Late stage

If the cake is selling, you really want to focus on the end goal: Is the café selling more? It could be that people buy this cake, but overall sales aren’t increasing, because this cake simply substitutes something else.