5 Steps To Measure The Right Things In The Right Way — Fit To Lead

If you measure what everyone else measures you will be on the pathway to commoditisation not differentiation and long-term success. Rather than hoping to capture the important things by measuring everything, leaders need to learn to select the critical few measures that will have the greatest impact. But once you have the data don’t let yourself down with poor reporting and/or interpretation.

If you believe measurement and data’s purpose is to help you improve performance then why are you seeing little or no improvement? Maybe it’s because you are ignoring the data and overly relying on your intuition ( overconfidence bias) or taking the easy option and selecting the easy to measure ( availability bias) or you are settling for the comfort zone of the status quo. None of which will take you to the next level.

As Michael Lewis describes in Moneyball, the Oakland Athletics baseball team with the second lowest payroll in baseball needed to develop their own way of winning. They redefined how winning is accomplished i.e. when players don’t make outs. In baseball three outs define an inning, meaning anything which increases the chance of making an out is bad and anything which reduces the chance of an out is good.

The general manager, Billy Beane, believed that traditional baseball scouts undervalued the players with high on-base percentages and overvalued other player attributes (e.g. physical build, footspeed, fielding ability). With the insight that a player’s capacity to get on base was a much better predictor of how many runs a player would score, Oakland Athletics bought winning players without paying up (as all the other teams continued to focus on different measures and their inaccurate gut feel). Oakland Athletics despite not having the financial firepower of the other teams went on to get to the World Series playoffs with this winning strategy.

Yes it’s important to measure the routine areas such as revenue growth (financial measures) and customer loyalty (non-financial measures) but this strategy is not sufficient for a sustainable winning performance. As we will see many of the same old measures have only a loose correlation with the objective of value creation.

If you measure what everyone else measures you will be on the pathway to commoditisation not differentiation and long-term success. Copying others gets you so far, but effective differentiation takes you to the next level.

Very few companies stop to think whether their measures are useful and are leading to better decision making or improved performance.

As Peter Drucker, the leadership guru, said, “What gets measured, gets managed.” So picking the right measures will be critical to business success.

This essay is a complement to the other essay I have written on performance measurement — Discover the power of effective performance measurement, the bad habits leaders fall into and how to correct them.

Step One — Articulate Your Key Value Drivers

Arguably the main goal of any business and the key to business success is value creation which means the value creation process should be and needs to be measured. But very few organisations know how value is created, let alone have the desire or ability to create a measurement system to support the value creation process. If something is not creating value, it will be destroying value.

The key value drivers of any business should come from your company’s strategy.

Your strategy, as discussed in my essay Craft Your Winning Strategy With 5 Key Questions, identifies your Winning Aspiration and your ‘ Where To Play’ (“ WTP”) and ‘ How To Win’ (“ HTW”) choices i.e. your theories of competitive advantage in the areas you want to play. This is followed by the Key Capabilities which are the activities, systems and processes you need to invest in and perform distinctively that bring your WTP and HTW choices alive i.e. range of activities you need to outperform your competitors and delight your customers. And then the company needs Management Systems that build and maintain the Key Capabilities.

Essentially in this step, you are trying to articulate your theory of how you will differentiate and win against your competitors i.e. your key drivers of value.

Step Two — Formalise Theories Into ‘If-Then’ Statements

Having defined your theory of winning and what causes your customers to buy from you, you need to develop measures which will best inform you about the linkage between your theories of competitive advantages and the results or outcomes you want to achieve.

The most useful measures reliably reveal cause and effect. If you don’t know the sources of customer loyalty, you can’t identify the metrics that will help you improve it.

Cause and effect measurements have two characteristics — predictive and persistent.

Predictive in that there is a causal (not casual as in informal!) relationship between the measure and the desired outcome. As discussed in my previous essay, good measurement is a proxy for the real thing otherwise it’s a waste of time.

Measures are persistent when the action consistently results in the same outcome.

To help formulate your theories of winning create a series of ‘if-then’ statements. This helps you figure out what evidence you need to collect.

Using an example of a restaurant business I used to be involved in. Our competitive advantages included family-friendly, affordable, family shopping locations with polite, relaxed, kid-friendly and clean-cut staff. We were experiencing same-store sales declines in a number of restaurant sites. We weren’t sure of the causes but amongst some of the feedback from focus groups were issues around cleanliness of the restaurants.

So we discussed various initiatives using the ‘If-then’ statements methodology.

Initiative I (50% of the sites):

• If we train team members on cleaning and what the site should look like at all times, then they will keep the restaurant cleaner
• If the restaurants are kept cleaner, then the customer will notice and evaluate the restaurants as cleaner

Initiative II (50% of the sites):

All sites:

• If the customer evaluate our restaurants as cleaner, then our brand image will improve and customers will likely choose us again
• If our staff members are evaluated as more polite and kid-friendly, then our brand image will improve and customers will be more likely to choose us again

Step Three — Develop Valid Measures For Your Theories

Having set out clearly your cause-and-effect theories, you then create the measures and targets you require to quantify the performance.

Developing a valid measurement approach for your cause-and-effect theories is probably the toughest part of this exercise.

You are translating your ideas about winning and differentiation into something which can be measured and analysed. You are translating qualitative thoughts into quantitative measures. It’s important to get your team members involved in developing the measures so that they feel part of the process.

You don’t want to measure everything. You want to focus on the few right things.

Investing your time and resources into measuring only a few ‘right’ things allows you to prioritise what to obsess about and gives you the discipline to deliver on your strategy.

After you have created a list of potential measures and targets, you want to evaluate them and filter them down to the best ones using these questions:

• How strong an indicator is this measure of the result?
• How feasible will it be to measure this?

I recommend you grade each measure on a scale of 1–7 for each criterion — strength and feasibility.

You also need to consider the bigger picture i.e. how different measures relate to each other — complement or conflict — between the different teams and business units. What might be the unintended consequences of measuring the result — positive or negative? What other areas of performance might be adversely affected as this result improves?

Continuing with the restaurant example aforementioned, we triangulated our existing restaurant cleanliness grading system with a combination of measures. We were interested in customers’ perceptions of cleanliness, not how many times team members wiped the surfaces clean. We hired mystery shoppers to periodically visit the restaurants and complete a survey about their experience covering cleanliness and service levels. We also asked the customers themselves. Each measure offered a different perspective.

Step Four — Measurement Reporting

Reporting can be the downfall of many a performance measurement process. Spreadsheet or dashboard applications often don’t work. They are hard to navigate with too much detail and information displayed in indigestible formats. They are often time-consuming to produce and rarely give the insights needed.

Here are 4 steps to designing effective dashboards according to Stacey Barr:

1. Structure in alignment with strategy. As everything should be coming from the strategy it makes sense to use headings in the report consistent with the winning aspirations, goals and priorities.
2. What, why and now what? A good report clearly displays the signals the measure is showing i.e. whether the performance is improving, getting worse or staying the same. The next part should provide answers to why you are seeing these signals. The final part is about exploring and deciding on the best way to respond to signals in order to achieve the target. Here are the three questions:

• What is performance doing relative to target?
• Why is it doing that?
• Now what do we need to do to move it closer to the target?

1. Use graphs that signal and engage. Use graphs that accurately and quickly show what performance is doing right now. Tips on choosing a graph include keeping each graph focused on one measure, providing the source of the data, explaining the scope of interpretation and the meaning of the measure.
2. Automate, automate, automate. Think about how much of the report you can automate. Measurement reports can take a long time to produce. The more you can automate the more of your time is freed up to analyse the data and act on it.

Performance reports need charts which show the results over time, highlight the changes in measures and indicate what results may be appropriate to improve performance. See below on XmR charts.

Step Five — Measurement Interpretation

You might think that there is nothing much to talk about when it comes to interpreting performance data but there is. Why? Because too often due to bad habits or ill-informed beliefs, our interpretation can lead us in the wrong direction.

The main culprit is what we compare our performance to. If we compare the performance to last month or quarter we are simply comparing two values to each other. The change between any two periods is a result of both ‘routine’ and ‘real’ change. Unless we know the size of the routine change (i.e. routine variation always occurs even when there is no real change in performance) we don’t know the real change. What makes last month an appropriate baseline to evaluate this month’s performance? What if last month was unusually good or bad?

You may think trend lines are helpful but without using R2 value which measures how well the trend line explains the patterns in a set of data, they are overly simplistic and often incorrect explanations of what performance is doing.

The surprising conclusion is that different comparisons can give you contradictory results. Scary! Lies, damn lies and statistics!

Statistical thinking is required for a good interpretation of measurement signals. Statistical thinking is about separating signals from random, routine noise. It’s about finding patterns to help us understand more and make informed decisions.

• performance has been achieved
• performance is improving at a rate fast enough that the target will likely be met
• performance is improving but not fast enough
• performance is stable and not changing
• performance is getting worse
• performance is unpredictable or chaotic.

But statistics do not give us exact answers. They give us probabilities and estimates which need interpretation. One effective tool which uses valid interpretation methods is XmR charts. You can use XmR charts for virtually any kind of performance measure. The two charts that make up an XmR chart provide the real signals of change and prevent us from focusing on the noise.

The different types of signals an XmR chart can help provide include:

Ultimately measuring the right stuff is about finding evidence about your key competitive advantages. The right evidence will give you insight into whether you are creating or destroying value within your company. The right evidence will help you find the key levers in your business which will have the biggest impact on revenue and/or profitability.

Measuring The Right Stuff Requires Effort But It’s Necessary If You Want Your Company To Get To The Next Stage

I am indirectly or directly involved in over 40 companies. Many of them have lots of data. Many of them use familiar metrics. Many of them are performing very well. But how many of them truly know what their key value drivers are and, if they do, how many measure their key value drivers effectively? Very few. You could say that’s my fault and to some extent that’s true! I should be coaching and advising them to get better at this.

I haven’t always had a positive view of measurement in the business context because seldom have I seen it improve performance. But measurement done the smart way is critical to better performance and ultimately business success. So remember these 5 key steps:

1. Articulate your strategic theories including your key value drivers
2. Formalise theories into ‘if-then’ statements
3. Develop valid measures for your theories
4. Skillful measurement reporting
5. Smart interpretation using XmR charts

If you want to learn how to find and interpret the right measures for your company reach out to me at mark@fittolead.net or via my website.

Originally published at https://www.fittolead.net on August 20, 2023.