How Metrics Can Actually Help
Taking cues from Andy Grove and John Doerr, my thoughts on how we can use metrics and goals for good, not for evil
The tale of the blind men and the elephant is a classic parable that holds within it a clear lesson: our capacity to fully understand something is often limited by our own subjectivity and experience. This adage is used all the time in the business world, and it can be easily applied to the rift between business and engineering when it comes to metrics and goals.
The problem is that engineers tend to see metrics somewhere between a bridle and a noose — a tool for control and punishment. Business people, on the other hand, tend to see metrics somewhere between a carrot and a Maserati — a tool for motivation and drive. Both views arise from different correct albeit partial perspectives on the broad topic of metrics and goals. So, before fighting about “who’s right” when it comes to metrics, it pays to understand the other side’s perspective.
TL;DR
Metrics and goals are primarily communication and alignment tools. When used for performance measurement, Goodhart’s Law states that they immediately sabotage their accuracy and effectiveness. To be effective, metrics and goals need to be set by the individuals responsible for moving them; they must also be completely under the individuals’ control. Finally, the cadence of review and re-evaluation of the metrics and goals by those same individuals determines their continued usefulness.
A Personal Story
I’ll start with a personal story from my first job out of university at a hardware company here in town. After working as an FPGA developer for about a year, I was promoted to the position of project manager for a new piece of hardware we were building. This particular project had a lot of software that was needed to manage the particularly complex hardware inside. This was my first experience managing a project and I was introduced to the Gantt chart for the first time. I built out the mother of all Gantt charts with over 100 little interrelated bars. Senior leadership loved it — clearly, I was on top of things! The problem was that in order to keep that chart up to date as the project moved forward, I had to spend 20 hours each week gathering feedback, updating the chart, and making the necessary downstream changes. Since that time, I have often felt that Gantt charts are evil by their very nature and that they were the problem. But, as with many challenges in life, the problem was a perspective that was too narrow.
What are Metrics?
At their simplest, metrics are quantitative measurements that provide insight into the inputs and/or outputs of a process. On the surface, metrics are fairly innocuous. Their hidden weight comes from the implication that they are indicators of what the business values. This implication results in obscuring the true goals of the business. The unintended consequences of this are clearly documented by V. F. Ridgway in his oft-cited 1956 journal article titled “Dysfunctional Consequences of Performance Measurements”:
Even where performance measures are instituted purely for purposes of information, they are probably interpreted as definitions of the important aspects of that job or activity and hence have important implications for the motivation of behaviour.
Ridgway’s paper discusses how performance measurement results in narrow job interpretation whether the metrics are single, multiple, or composite in nature. One example he uses is the centralized planning employed by the Soviet Union in its attempts to optimize factory efficiency and output. This is a fascinating subject on its own as Soviet workers were truly geniuses at gaming these performance management systems with comical results (e.g. You want to measure the weight of nails we make? What if we made 100 lb nails instead of the normal sized nails that most people need?).
The phenomenon discussed in Ridgway’s paper were later formalized by British Economist Charles Goodhart in 1975 and referred to as Goodhart’s Law:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
So the challenge is that measurement itself isn’t bad, but the problem arises when management declares it is going to measure something, thus initiating a game with employees to optimize that metric (or set of metrics) rather than serve the customer to the best of their ability. The most common instance of this game is the one where the business selects a deadline for delivering a project and then focuses metrics around that deadline. The result is invariably lower quality delivery.
The Genius of Andy Grove
Andy Grove was the first employee of the Intel Corporation in 1979. He went on to be the President and later CEO of the company, leading it through some of its most innovative and successful years of business. He was a gifted manager who well understood the challenge of metrics and goals, leading him to create something he called Objectives and Key Results (OKRs). In his book High Output Management, Grove lightly touches on the OKR, but most readers would be forgiven if they missed the importance of the concept being introduced. Jim Doerr provides a much richer description of Grove’s OKR system in his book Measure What Matters. The key insight of the OKR is that the employee chooses the metric along with the goal (Key Result) to fit within the organizational goal (Objective). This cleverly sidesteps Goodhart’s law by avoiding management choosing the metric!
Many business books contribute to the lore of difficult goals and their ability to stretch those who set them. Jim Collins calls it a Big Hairy Audacious Goal (BHAG) in his seminal book Good to Great, while Doerr refers to this trait as “stretch” in Measure What Matters. Doerr goes on to discuss how Google famously sets both “committed” and “aspirational” OKRs with the aspirational ones being only “70% achievable”:
At Google, in line with Andy Grove’s old standard, aspirational OKRs are set at 60 to 70 percent attainment. In other words, performance is expected to fall short at least 30 percent of the time. And that’s considered success!
(p. 139)
The entire idea behind this stretch aspect of an OKR is to encourage people to aim higher than they might normally consider. The question that one should ask though, is “Why does a stretch goal improve performance?” Doerr lays at the answer as follows:
Setting specific challenging goals is also a means of enhancing task interest and of helping people to discover the pleasurable aspects of an activity.
(p. 134)
This is an important piece of data because it explains that the stretch goal serves to make a task more interesting or to pique the person’s interest/curiosity. It bears repeating that this only works when the employee chooses both the metric and the target as the Key Result. If management chooses the metric or the target, the benefit is nullified immediately and Goodhart’s Law dominates. The key component here is to trigger a person’s natural desire for Autonomy and Mastery to drive performance.
Curiosity’s Role in Motivation
One of the particularly vexing things about performance is that a person will push themselves harder when there is nothing monetary at stake. The human motivator with no limitation is actually the desire to play or to satisfy one’s curiosity. The desire to learn one’s limits and improve one’s skills is an innate part of human nature. Daniel H. Pink describes this quite clearly in his book Drive. He points out that people are motivated to action in three ways:
- Extrinsic (Biological) — The desire for food, sleep, water, and sex motivates behaviour.
- Extrinsic (Reward & Punishment) — The desire to provide for one’s family.
- Intrinsic — The desire for self-actualization (Mastery, Autonomy, Purpose)
Interestingly enough, Pink points to research on the topic that shows how different types of motivation affect task performance. The research shows that creative tasks like Engineering tend to see lowered performance when motivated extrinsically and increased performance when motivated intrinsically. If you want to trigger intrinsic motivation, his suggestion is to focus on the following three areas:
- How employees organize their work
- How employees think about their work
- How employees do their work
The more autonomy employees feel in these three areas, the more strongly they will be intrinsically motivated. If we let employees choose their own metrics and goals that they can directly influence, let them instrument those metrics, and give them control over their approach to achieving the goal, we’ll maximize intrinsic motivation.
Metrics: Tools for Communication & Alignment
Coming back to the original problem, even though engineers and business people tend to see metrics and goals very differently, we now have the tools to broaden our perspective to both understand the conflict and resolve it. We need to see metrics and goals as tools for communication and alignment rather than for performance measurement and management.
Business people need to see that setting another person’s metrics and goals (Key Results) rather than letting them choose their own within an area of focus (Objective) results in decreased performance and a reduced sense of ownership and accountability. When a person sets their own Key Results, instruments the measurement of them, and feels free to adjust those Key Results when new information comes to light, that person is empowered to provide transparency into what they think is important. Understanding what employees think is important can be used by management to better understand the state of the business and to spot opportunities to clarify Objectives to achieve better alignment.
Engineers need to see that choosing metrics and setting goals (Key Results) within a corporate strategy (Objective) allows them to bring focus to their work while not losing sight of the ultimate goal of the business, which is to meet the needs of its customers. By viewing these Key Results as opportunities to challenge themselves while remaining agile with respect to changing assumptions and conditions, goals and metrics can be an effective defence against imposter syndrome and a way to discover purpose in their work by connecting their daily activities with the organization’s higher-level goals (Objectives).
