Metrics need to add value or they’re just a waste of numbers
Any freshly minted MBA will tell you “What gets measured gets managed” a quote often misattributed to management consultant Peter Drucker. Everyone is obsessed with metrics, measuring every element of a business and making decisions based on this data. In the era of “Big Data”, this is even truer. And I’m not here to say otherwise, exactly. I’m a big fan of calculating, tracking, and listening to metrics. But I’ve seen them misused often enough that I want to throw a cup of skepticism on the fire of metrics: not enough to put out the fire, but enough to create a smoky cloud of uncertainty. The fire will keep you warm and scare the wild beasts that roam the forest, but it won’t pitch your tent or clean up after dinner.
Precision is how many digits are in the answer and accuracy is how close to the true answer your estimate is. Think throwing darts: being centered on the bullseye is accuracy, being in a tight grouping is precision. A tight grouping far from the target is an example of precision without accuracy. In your high school physics class you might remember getting problems like “The distance is 1.7 meters and the time it takes to travel that distance is 3.2 seconds. What’s the speed?” If you answered 0.53125 m/s, you were wrong because you included too many significant figures, you had precision without accuracy. The correct answer is 0.53 meters/second. This is such a pet peeve of mine, that when I would see examples and ask my 4-year son what this was, he would roll his eyes and say, “Precision without accuracy”, with the tone of a bored teenager.
We sometimes forget that all measurements are estimates. There may be biases in our measurement tools (e.g. a scale that hasn’t been properly tared), sampling errors (e.g. we only asked people at a Sierra Club meeting for whom they’re voting), or uncertainty due to sampling size (e.g. we made five measurements). For whatever reason, the metrics we work with are imperfect, and must be treated that way.
I once worked at a company that was obsessed with metrics, but didn’t understand the difference between accuracy and precision. In a report I stated that a process was successful at least 90% of the time. My manager told me that I needed to be more precise. I could have calculated an answer of 93.457%, but the accuracy was such that I could only state > 90% with confidence. I could have spent another month gathering data to get the higher precision answer, but that was also unacceptable. The focus on precision instead of accuracy drove me a little bit bonkers.
For every metric you collect ask yourself how this measurement will impact the way your company is managed. And for every decision, you should ask if there are useful metrics that can inform your decision. An example of a useful metric is how busy each of your teams are. For teams that have a large backlog of work, it might be time for to add to the team. For a team with very little work to do, it might be time for a layoff or for the sales team to focus on that kind of work.
Metrics that drive stupid actions are even worse than ones that aren’t actionable. In a factory you can measure how long each step in the manufacturing process takes and you can work to drive that time to be as short as possible. If you focus on that, but ignore quality or employee turnover, you’re not going to be optimizing the factory. Don’t focus on the simple metrics at the expense of the important ones.
Important issue likes building a culture of innovation, employee and client happiness, and reputation are difficult or impossible to measure. One of the reasons that companies focus on quarterly profits rather than long-term value, is that it’s easy to measure quarterly profits, but who knows what the future holds. If your CEOs compensation is based on metrics that don’t align with the long term interest of the stockholders, employees, and customers, don’t be too surprised if he does things that will help those metrics rather than the company.
Metrics are valuable, but being a leader is not about blindly following metrics, but using them to inform your actions, just like your experience and intuition does. If your gut is telling you that an action motivated by a metric isn’t wise, then dig deeper. Don’t ignore either your metrics or your judgment, but understand why they are pointing in different directions. That way leads to greater wisdom, better metrics, and a healthier company.
Originally published at https://asilberlining.com on November 3, 2016.
