Blind Values, Empty Metrics
Nothing compares to great data. Nothing does more to help me clarify situations, develop solutions, or make changes. And when data is woven into an honest narrative, it makes for some of the most beautiful, compelling expressions of thought we can find. It reminds me of a great line from Steven Pinker in his seminal work, Better Angels of Our Nature:
Narratives without statistics are blind, but statistics without narratives are empty.
Weaving narratives comes natural to most of us. But statistics has to be learned.
This is a great way to level up. So I explore a lot of books of data. But it isn’t easy for generalists like me. Most resources are textbooks and instruction manuals that teach K-Means and logistic regression and all that jazz. Some of which is intuitive. Some of which isn’t. Developing these skills can feel great. But then come the questions:
How do I apply this to my work?
What, exactly, must we look for in our situation?
What questions should I pose?
What data should I measure?
In other words, despite all my love for the practice, I occasionally fail to understand how these methods help us run a better operation.
New Techniques For Old Questions
At the core, data is the resource we use to answer the age-old, constant conundrum: what do we change and what do we maintain? We ask ourselves this question all the time. And if that weren’t enough, we often change our answers as we change our goals.
Consider it from another age-old, constant question: what are we going to eat for dinner?
If you have a busy schedule, you probably find yourself modifying your weekly dinner plans from home-cooked three-course events to quick and easy dishes made in 30 minutes or less. Rachael Ray built a whole franchise on this. All in an effort to simplify because hectic schedules require us to solve for a single variable: time.
This works for a while. Then it gets boring. Eventually, you find your food is somewhat bland or it’s all frozen, unhealthy stuff. Simplifying saves time but diminishes other factors. Our tastes change, we adapt, and we’re unsatisfied again. So we change once more, bringing healthier foods and different recipes into the mix in an effort to now optimize on three variables: time, freshness, and taste.
Which, coincidentally, tends to add more time in the kitchen. Which is what you wanted to avoid in the first place. So a cycle is born. Simplification to optimization to simplification, etc.
Personally speaking, my approach to weekly cooking has changed drastically through the years. All along this continuum. It’s maddening and I’m about ready to just sign up for Soylent instead.
Jokes aside, data is the critical resource throughout this exercise. ROI is determined by the time and money invested in the kitchen divided by the satisfaction at the dinner table. When are these things just right? Honestly, no one really knows.
Because the data is very fuzzy. We measure by feeling. We go with our gut. Literally. And this is perfectly understandable in our personal kitchens and dinner table experiences. But going with our gut in the workplace feels a little less professional.
There’s a better way.
A Lean Pursuit
Again, the bedrock questions that we all face in any regular, goal-oriented work include:
What do we change?
What do we maintain?
And we respond with answers that either simplify or optimize but rarely do both.
How do we know which path is best?
I think the best method for answering these granular decisions is the deliberate, iterative cycles of the Lean process. The process and rationale itself is covered in my review of Eric Ries’s terrific work, The Lean Startup. That book is the right place to start.
But if statistics textbooks leave you wondering how to bring new analytical techniques to work (when, exactly, do I need polynomial regression?), Ries’s Lean Startup can leave you wondering how to apply data to the process.
The next resource to bridge that gap is Alistair Croll and Ben Yoskovitz’s Lean Analytics. At times, it is a wide-ranging treatise that offers soft illustration on a given topic. At other times, it is a deep, focused examination of fundamental concepts. And throughout, it is the book that shows me what analytics is really for. Specifically when it comes to actual metrics.
The One Purpose of Metrics
Early in the work, our authors state something so obvious it can easily forgotten:
A good metric changes the way you behave. This is by far the most important criterion for a metric.
Again, that feels obvious until you think about the number of times we are all seduced by the numbers that do not change the way we behave (e.g., “likes” on social media). The distinction here is between vanity metrics and actionable metrics. As our authors explain:
Vanity metrics make you feel good but don’t change how you act. Actionable metrics change your behavior by helping you pick a course of action.
We know that some of these metrics are quantitative. Some, of course, are qualitative. When we go back to the weeknight dinner example, the quantitative metric is time spent in the kitchen. It is an objective measure. The qualitative metric is our level of satisfaction with the meal. This is deeply subjective and open to broad interpretation. Especially when I try to make Thai food.
I think what matters is that both types of metrics show what we value. In fact, actionable metrics are just a more precise definition of what’s important to us. It may be the only way to really define what’s important to us in ways people can understand.
After all, what do we mean when we say we “value” the customer? There is a philosophical component to this idea but a measurable definition is needed if we want to consistently demonstrate the ideal. It starts with the easy correlation to satisfaction ratings. A target satisfaction rate of 95% goes a long way to explaining how much you value the customer.
But there are other metrics, too. How much of the annual budget is invested in customer service? What does that yield in Customer Retention? Or Total Customer Cost?
These questions are a means for clearly articulating your philosophy beyond empty platitudes. Maybe “customer retention” matters. Maybe it doesn’t. It depends on your intrinsic values.
This work shouldn’t be too formulaic or deeply complex. It shouldn’t be without measure, either. To gently adapt the Steven Pinker line:
Values without metrics are blind, but metrics without values are empty.
So great metrics lead to action. As it is aligned to your values as a service or business or kitchen cook.
An Extra Benefit Of Metrics
And ultimately, it should lead to simplification, too. Or rather, focus. Because the one bad thing about metrics is that you can have a lot of them. In Lean Analytics, the authors give a basic overview of all the different measures that you can find in any major technology sector. There are hundreds. And that just barely scratches the surface.
You can’t have them all. The authors recognize that a collection of many equally-weighted actionable metrics typically leads us to having no action. That’s not exactly the “Lean” way.
Optimization, and school valedictorians, would say otherwise. We can get an A+ in everything! But that doesn’t work until you’ve simplified to a single metric above all of it. In the school valedictorian’s case, that single metric is GPA.
What is it for the business? The service? The weekly meal plan? I’m not sure. But I want to know. It’s amazing what can happen when we get it right. It’s terrifying what happens when we get it wrong.
In closing, I don’t want to measure everything in my life, per se. I just want to define what matters most. A solid measure can help me do that. And thus help me simplify.