Framing Metrics for Success

“Measure what matters” — John Doerr

For many Organisations there is a tendency to see value in measuring every and anything that is measurable, without investing time evaluating each metric to understand the why, the value it provides and the potential consequences.

Metrics are regularly used as ways to control outcomes and behaviors, the more metrics the better! I routinely see metrics chosen without consideration of their possible benefits, negative side effects, or their relevance to their target audience.

Some examples of metrics and measures that I’ve seen to encourage or drive particular outcomes and behaviors include;

1. Incentivising individuals on revenue goal — with the desired intent of driving sales to improving the company’s available cash.
2. Goal developers on lines of code produced — with the belief this will increase productivity and speed of value to customers.
3. Reward (or punish) teams by focusing on error rate — to ensure quality.

At first glance these seem simple, sensible and logical metrics to measure and they WILL control behaviors but maybe not with the intended results. Let’s have a quick look again at each of the above

1. The behaviour I saw was people started making less profitable and higher maintenance sales which eroded margins, hence reducing available cash
2. Developers started cutting and pasting large amounts of code without thinking about the longer term maintenance, testability, readability, or quality of the code, reducing the overall value delivered.
3. Measuring errors resulted in a slowdown of work done and a reduction in outcomes released from teams as new inbound work was rejected on the basis it required more discovery or elaboration in order to reduce errors;

There is a lot of information about metrics, and observation systems that let us collect metrics and measure just about anything: people, teams, leaders, sales, e-commerce performance, business performance, finances and the list goes on.

The challenge is not that there aren’t enough things to measure, it’s ensuring we are using and measuring the right metrics to accomplish what we are trying to achieve. How can we be consistent on the type or value of the metric, how do we attempt to detect a metric that cause bad behavior and how do we create a culture where metrics are used for continuous improvement and achieving.

When thinking about a metric I strongly suggest an upfront discussion clarifying the language used, ensuring a common understanding of measures, the why and the expectation. Share this information throughout the organisation to ensure everyone is starting from the same solid base. Noting that this is not about the actual metric but its definition, intent and scope. My strong recommendation is to start framing up metrics with the template below to create the base.

Framing a Metric

When thinking about a metric consider these seven dimensions, this will help to make them:

Valuable

• There has to be a clear and directly associated relationship between the metric and how it will achieve a better outcome. Without this a metric should not be measured as it becomes just because something is measurable metric.

Achievable

• If a metric is going to be used to help drive goals, then ensure the goal is honest and achievable. This doesn’t mean that they need to be easy. Setting up unrealistic targets will create defeatist attitudes, while too easy only the minimum effort to reach them will be made.

Realistic

• If the metric is going to be created and measured, then it has to be either within a person’ s or team’s control or influence. Metrics outside this zone provide no or negative value as they will shift independently of an individual or team’s actions and deflate morale.

Real Time

• The data needs to be as real time as possible, with easy accessibility combined with high visibility. Timely metrics allow for effective learning, ability to quickly change course and minimise the distraction of reporting. Metrics that are slow to collect and disseminate, e.g. are shared monthly/quarterly/yearly provide data that’s often too late to act on.

Opposite

• Care and thought needs to be applied to the potential outcomes or behaviors of the metric. The metric may create opposition to the goal trying to be achieved. As in our example 1, increase revenue could lead to selling that reduces margins than potentially profitability.

Observer

• For many reasons it is important to understand who the observer of the metric will be, the metrics may all be visually available but must directly speak to the metric’s audience, eg even though a team’s test time metrics may be seen by anyone, they provide the highest value to the team who has an understanding of their pipeline and the work, they could be useful to others consuming the services from an engineering space, however potentially significantly less value or understood for someone in logistics or finance.

Measurable

• Metrics have to be both measurable and as low cost (ideally zero cost!) to measure as possible. This implies automated collection, reducing or eliminating the burdens to collect and administrative overheads. Over and over I see manually collected metrics that quickly become out of date, and often include errors or introduced biases by the human collector.

Once there is agreement and alignment on the framing of a metric, it’s time to decide on which metrics to use. I have found that it is best to start by creating two or three categories, even if there is only one or two metrics in each category, then collect some data on these metrics. Also make sure you have captured you framing outcome so that others may quickly understand the metric and your intent. An example of three categories could be Technical or Delivery Metrics, Customer or Business Metrics, and then Team Metrics.

Remembering that you can add more metrics and information about your metric as you learn, as well as remove metrics that fade in relevance, are not realistic, drive unwanted outcomes or no longer measurable. Actually, collecting data is important as it helps to make the metrics real, which in turn evolves the thinking of the metrics purpose and collection process early during their creation. A common trap is for teams or entire business units to get stuck in excessive planning that gets wasted once data gets collected and fails to meet the expectations of the plan.

My goal is to also share some examples of framed metrics, the need to pair metrics, displaying metrics as well as some ideas and metrics I have historically used.

I hope that the time you have invested has returned value to you and encourage you to provide real-time and measurable feedback! A comment or simple thumbs up lets me know how I’m doing.