The Liberators
Published in

The Liberators

Stop Measuring The Pizzas And The Cooks

Why output-oriented metrics are mostly a waste of time in software- and product development

“Instead of relying on assumptions, intuitions, and expectations, rely on data to improve how you do your work.”

Metrics for pizzerias

On face value, a couple of metrics might suggest themselves. You could measure the number of pizzas produced on a night. More is better, right? Another option is to measure how many people are in your kitchen and how much time they spend working there. Again, more would seem better. And how about the number of people walking into your shop? The more you have, the better you are doing.

Except …

None of these metrics are helpful as they are only concerned with output. They measure only how much work is done. They don’t tell you anything about the value of that work; the outcome. You could do really well on all of these metrics and still run your shop into the ground. You can create hundreds of pizzas every night. But without customers to buy them, you have only created a heap of waste and lost money producing it. The same happens when you have dozens of people working in the kitchen, putting in many hours of overtime. Without actual customers, you’re losing a lot of money and generating waste. Even when your shop is succeeding in attracting hundreds of people, your shop will still go belly up if they don’t buy anything.

“Output-oriented metrics tend to encourage the accumulation of waste.”

So, measure outcomes instead

If you want to reduce waste, a much better approach is to measure the outcomes of the work. How many pizzas are you actually selling? What is the margin of profit on each pizza? How happy are your customers with their pizzas? How many people return for more? What percentage of the people walking in actually sits down to order? How rapidly are customers being served? What is the ratio between how much you spend and make for each week, month, quarter or year? How many pizzas and ingredients do you throw away at the end of a night?

It can be harder to measure outcomes …

Now, outcomes are often harder to measure than output. Measuring outcomes requires a good understanding of what your value is and to what degree that value is reaching your customers (in itself a good sign that these are the metrics that matter). Given the difficulty, a concession might be that you’ll just have to do with output-oriented metrics because that’s what you’ve got. An underlying assumption here is that output and outcome are related, meaning that if you can at least measure output, you’ll have a good sense of what your outcomes are even when you don’t measure them.

Back in the world of software development …

So if it makes sense, why is it that we’re obsessed with output-oriented metrics when it comes to software development? Measuring how many items or story points are completed per Sprint is the same as measuring the number of pizzas produced. It just doesn’t help. Tracking how many people or teams are working on a product and how many hours they are putting in, is just as effective as deciding how successful your pizzeria is by counting the number of cooks in your kitchen.

“Measuring how many items or story points are completed per Sprint is the same as measuring the number of pizzas produced. It just doesn’t help.”

Coincidentally, it seems to me that the strong focus on output-oriented metrics is a huge driver of scaling efforts in many organizations. Without considering how much value is actually being generated, adding more people and teams seems like the way to go to generate more value.

Metrics for Responsiveness

  • Lead Time or Cycle Time: how much time transpires between when a need is identified (lead time) or when a team starts work on it (cycle time) and when it is released to stakeholders?
  • Work-in-Progress: how much work is ‘in progress’ at the same time on different levels of the system?;
  • The number of interruptions: how many times are Scrum Teams interrupted from their focus during a Sprint?

Metrics for Quality

  • Code Quality: the quality of the codebase with static code analysis, test coverage or whatever makes the most sense;
  • Total defects: the number of known defects in your software;
  • Stakeholders happiness: how happy the stakeholders are with the quality of what is being delivered to them;

Metrics for Improvement

  • Team Morale: how willing people in your team and its stakeholders are to collaborate together, even when the going gets tough;
  • Innovation rate: the percentage of time spent on new technologies, skills, and approaches;
  • Number of dependencies: the number of external dependencies that teams have to deal with in order to get work done;

Metrics for Value

  • Return on investment: the financial value of the work that is done by the team versus the cost of creating that value;
  • Conversion rate: the number of visitors and people that turn into paying customers (not always relevant);
  • Profit margin or revenue generated: the profit margin on work done or the revenue generated by it (depending on the kind of company);

Closing words

Stop measuring the proverbial pizzas and cooks in your work. Tracking metrics like that only tells you about how much work you’re doing, but not how valuable that work is. For all you know, you’re creating hundreds of pizzas with dozens of cooks that nobody wants; its a complete waste. And that is the underlying problem of output-oriented metrics. They encourage the creation of waste in your process if you don’t consider the bigger picture: value. And that is why it makes more sense to measure the outcomes of your work. Are customers happy? Do they pay for what you create? Do they return? In this post, we shared some examples. Not just for pizzerias, but also for software development. Give it a try, and see what happens!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Christiaan Verwijs

I liberate teams & organizations from de-humanizing, ineffective ways of organizing work. Passionate developer, organizational psychologist, and Scrum Master.