Why OKRs are broken and how can we fix it?

Everyone needs some way of defining what is important. What should a team work on? As an engineering manager one of your principal tasks is to help create alignment and direction.

OKRs have become a wildly popular objective setting methodology in the last decade. Initially, I was a fan. OKRs created better clarity with less paperwork in comparison to what came before. However, lately I am seeing the flaws in the methodology or how it is applied in practice leading me to the conclusion it is broken.

I want to throw away the damaging aspects, extract the beneficial aspects and evolve it into a new methodology that is more effective.

To introduce we are going to discuss the flaws then introduce an evolution of OKRs that I call COM — Context, Outcome and Metrics.

Whats the problem with OKR?

Flaw #1: What not Why

OKRs launch straight into the Objective — the ‘what’ needs to be achieved. OKRs often fatally assume the team knows why this objective is important and how it fits in.

Let’s take a sample ‘O’ from a site teaching OKRs (in reference to an offensive coordinator in a football game).

Seems obvious. Generate more meters of attack — presumably to win? But really why? Why passing attack? Why 700m specifically?

OKRs are often described as “I am not going to tell you how to do it (not micromanage) but take that hill”. However, before you go sweat and bleed to take that hill do you understand why we need to that that hill?

To solve we need context. As Simon Sinek would say — start with why. An example of missing context:

Context: Last year we analyzed our wins and losses. Our defense and rushing game is good/average. However, we are 17/20 in the league passing attack. If we upped our passing attack to 700m we would have scored 180 more points taking us to top 5 in the league

Now we can connect with that objective, get motivated and get behind it.

Anecdotally, as an engineering manager what I have found is that taking 5mins, 10 mins or even the entire meeting to share context is the best use of time whenever context is not widely shared. I have seen a hundred solutions where not quite the right thing was built because of the wrong requirements based on a flawed understanding of why.

In COM we start with context.

1. Why are we doing this? Context is the #1 piece of information to share with any team before discussing anything else — requirements, design, prioritization.
2. With good context a good team will self-organize and solve everything else. Without it, they are searching in the dark. How many organizations have really solved the gap with most of the organization really knowing what is going on.
3. Any engineering leader should have as a mental reflex explaining the context in any meeting before going into anything else — requirements, design, prioritization, etc.

Flaw #2: Objective is a Poisoned Word. Outcomes is Not

Language is important and powerful. Objective is a poisoned word based on the series of objective setting methodologies inflicted on employees (anyone remember SMART?)

Furthermore, Objective in OKR despite being about outcome is often output/‘what we are going to do’. In the football example — the ‘objective’ was ‘pass 700 meters’.

Is pass 700m an outcome or an output?

At the same time product management thinking is leading the transition from Outcomes to Outputs. My experience is the response to “What is the outcome we want to achieve?” is more effective than Objective. Outcomes is more descriptive, free of the baggage and leads the conversation to what will be different in the world.

In COM we propose to follow product thinking and drop objectives. Let’s talk in outcomes. In our football example the outcome would be:

Outcome: End the season in the top 5 by winning more games with an improved passing offense delivering 700 yards a game.

Hopefully, the change in context and outcome is already more appealing than the OKR version.

Flaw #3: OKRs Conflict with Agile and Innovation

Most organizations are still working to get everyone to truly ‘be agile’ (being agile versus just ‘doing scrum’ is a story for another time).

However, what happens when OKRs hit Agile? To paraphrase a conversation with an engineering manager who was attempting to reconcile the two:

Help me understand — we want to ‘be agile’ to respond to change over following a plan and seek value vs lock down scope. But at the start of the quarter OKRs require us to write down the outcome and quantifiable key results on how success will be measured?

Where Agile emphasizes responding to change over following a plan OKR tries to lock down up front the Objective and how to test if the objective was met (the Key Results)!!!

Critics may say that the OKR methodology is not antithetical to agile. That OKRs can evolve. However, the relationship is strained and ugly. We need to give up on trying to define success up front.

A central tenet of COM is that it does not seek to specify success up front.

1. COM is instead a tool for creating clarity. We set the best direction we can at the start.
2. We expect direction to evolve as the problem reveals itself.

Flaw #3: KR Measure the Result not Progress and Causation

Let’s return to our football example on OKRs.

The good part of KR is that is generates a conversation on how will we define success. The bad part, as we have discussed, is that it attempts to lock down success up front.

In COM we ditch KRs for Metrics. We define Metrics to be:

1. A causal factor in the outcome we are trying to achieve
2. That we can quantifiably measure
3. That we will put in place automated measurement at the start of the project and make these metrics visible to the team
4. We are more interested in knowing/measuring and improving than setting artificial goals
5. You may set lofty aspirational goals for any of these metrics. However, it is accepted that in projects with high degrees of uncertainty this is a aspirational/motivational tool rather than an engineering goal

The metrics for our football example would be:

1. Metric 1: Passing Attack Meters Achieved per Game and Average
2. Metric 2: # Passes, Completion %, Interception %
3. Metric 3: Shots made/Successful/Missed

Please take our now complete COM and compare to the original OKR. Which one makes more sense to you?

Flaw #4: OKRs have become owned by management and intertwined with Performance/Comp.

One of the aims of tech culture is to not micromanage. To not ‘hire bright people and tell them what to do’. OKRs can be implemented well and setting them is done in collaboration with the team.

However, often they are implemented in a waterfall manner where management sets the ‘O’. In this scenario you have removed the team from innovating and validating the ‘O’. In the worst case, you have a team free to unleash their creativity and innovation on fundamentally a bad objective.

OKRs also have an unhealthy relationship to performance. Whether it is admitted or not it is often believed there is a strong relationship between OKRs and Performance at the team or individual level. However, this is unfair given they were written before the problem was fully understood. This leads to a tension and fight over the OKRs as they relate to measurement of performance or the team/individual. In the worst case we see optimizing for the KR versus real business value.

In COM we

1. borrow from the lean mantra “everybody all together, early on”. In COM the entire team participates in all aspects of the methodology.
2. Disconnect COM from performance. Engineering managers should reward on velocity of discovery, quality of solution produced and engineering decision making. Engineering managers should be able to have enough examples of this to have a performance conversation without relying on the OKR as the litmus test of performance. Disconnecting the two unburdens the goals and alignment from performance.

Okay Okay. What is the Replacement?

The proposed replacement is a methodology COM comprising

1. Manifesto
2. Three Elements
3. Practices / Ceremonies

Manifesto

Creating goals, clarity and alignment is crucial to achieving success. However, we tackle complex problems where at the outset the solution (or problem!) are not fully exposed. COM is a people orientated clarity and alignment methodology influenced by agile to be clear on why we are doing what we are doing, setting direction, creating clarity and evolving as we learn more. We believe:

• Respond to change over follow a plan: Direction is required but it changes as our understanding of the problem changes. In the same way we can not design up front solutions we can not design up front measurements of success
• Everyone is part of the same team. There is no manager and employee. The team owns the goal setting methodology and can iterate/continuously improve. It should be a topic of retrospectives vs fixed.
• Value: Organizations should always reward delivery of value, quality decision making and learning. A goal methodology can support this process but it can not and should not be codified.

Three elements (COM)

• Context: What is going on with the industry, our company/division. What is the landscape — what are our opportunities and threats. What is our approach to succeed?
• Outcome: What will be different in the world after we are done?
• Metrics: How can we measure progress to this goal?

Practice

Teams should self-organize their own practice. A hypothetical starting practice could be:

1. The combined team get together to discuss context and have the opportunity to ask questions. Tools like mind maps to explore / capture the context are encouraged. Any discovery required is performed
2. The combined team draft the outcomes
3. Metrics are identified which help measure factors and progress towards the outcome
4. Before work begins metrics are gathered, implemented, automated and baselined.
5. As time progresses and the problem reveals the team adjusts any aspect of the three elements.
6. At any stage the team looks for continuous improvement in the entire process