If we know in which direction we want to go, does it matter where we are?
And for that matter, to where exactly we’re trying to get?
Continuing to reflect on some of the topics that I’ve covered in the last couple of posts, I want to spend more time today on numerical measurement and its alternatives.
While the answer to the question posed in the title may seem like a resounding “yes”, I’d like to suggest that in some circumstances, it may actually be “no”.
To be clear, I’m not a measurement or metrics hater. And this is not a rant about metrics. But given that I’ve spent quite a bit of time calling out their deficiencies, I owe it to myself and others to offer alternatives that are not meant to completely replace them, but to expand the toolbox so we can use the more effective tool to the problem at hand.
I won’t repeat my whole case on the challenges with numerical measurement but will just briefly mention that they typically elicit some non-trivial “operational” challenges in both the collection and interpretation of the data. And perhaps more importantly, they also pose some more “strategic” challenges — they reduce a highly complex reality into a very simplified representation. Sometimes that’s incredibly helpful — separating signal from noise and creating clarity on what’s truly important. But often times over-simplifaction leads to solutions with both cognitive and behavioral flaws when applied back in the complex reality.
The alternatives to this approach depend on the purpose we were trying to accomplish with numerical measurement begin with, something that I’ve noticed I haven’t given enough attention to in the past. Introducing some distinctions there helps to identify viable alternatives, at least in the two cases outlined below.
Measurement to articulate direction
One common use case of numerical measurement is to articulate direction. By describing where we are right now and where we want to get to, we implicitly define the direction in which we want to go:
- We want to get from point A to point B = we need to drive in direction C
- We want to improve margins from 13% to 15% = we need to improve margins/become more efficient
- *nerd alert* describing a vector using the coordinates of its start and end points
Often times the start and end points are rather meaningless in and of themselves. It’s the direction or delta between them that matters.
Yet describing the start and end point are not required to describe the direction. I can still “drive south” without saying “get from SF to LA”. Not the perfect example, I know, but hopefully still gets the point across.
Measurement to choose between options
Another common use case of numerical measurement is to choose between options.
For example, choosing what driver to focus on next in order to improve employee engagement. This is often done by having employees rate each one of the drivers using a Likert Scale, translating each rung in the scale to a numerical score and sorting the drivers from the lowest rated to the highest rated or from the one that worsened the most to the one that improved the most.
The distinction between cardinal and ordinal utility can help us find an alternative. We don’t even have to go into the debate about the feasibility of truly measuring the cardinal utility of each one of the drivers and simply say that since we’re only using the measurement to choose between the options, the ordinal utility is sufficient. And in that case, there’s a simpler alternative to numerically rating each one of the drivers in isolation: asking employees to stack-rank the drivers from the one we should focus on the most, to the one we should focus on the least.