Thoughts About DORA Metrics and Cars
Ever wondered why DORA metrics are so powerful? And what is the connection between them and a car’s brochure?
Introduction to DORA Evaluation Metrics for Development Teams
Based on 6+ years of research and data from more than 31,000 professionals worldwide, Google Cloud’s research organization, DevOps Research and Assessment (DORA), has identified key metrics that indicate the performance of a software development team. Like many development organizations, our team has adopted these famous four “DORA metrics” — Deployment Frequency and Lead Time for Changes (which collectively measure velocity), along with Time to Resolve and Change Failure Rate (which collectively measure stability) — to help benchmark and continuously evaluate our collective efforts.
According to the research organization, development teams that measure these values and continuously iterate to improve them can achieve significantly better business outcomes.
The highest performing teams across all DORA metrics — known as “Elite Performers” —
“are twice as likely to meet or exceed their organizational performance goals.”
But why is that? What is so special about these metrics, and how do they provide such an accurate analysis of team/organization performance?
To answer these questions, it’s important to dig in to what these metrics are actually measuring and how more traditional KPIs like predictability, throughput and quality aren’t as strong in predicting organizational delivery performance.
What are metrics for?
First, I need to answer a more basic question: why do we need metrics? What is the purpose of measurements? Why do we measure anything? Let me try and do that by using an analogy.
The team <-> car analogy
Let’s imagine our development team is riding a car.
Instead of taking on a development task, planning how to get it done and then implementing it, they take a task to get from point A to point B, plan the route and then ride.
So, let’s imagine you are part of this team. You’re focused on getting to point B. You’ve planned your way and started your journey. Let’s say you are about halfway there, and what you see is this:
You’re left with no directions or indicators.
You’ve lost your map, have no clear sense of arrival time — and don’t even know if you have enough power to make it to your destination.
How does this make you feel?
Having a dashboard
Now let’s imagine you have a dashboard, and it looks like this:
A dashboard gives you the answers to critical questions about your journey. You now know whether you are going to make it or not.
And that’s exactly why we measure.
We need to be able to understand where we are, what is coming our way and whether we can make it to our destination.
Just as we need a dashboard in our car, we need data-driven measurements for our development team to understand how to get — and stay — on the right track, and optimize the quality of our work.
We know now why we measure in general, but what, specifically, is the purpose of measuring the DORA metrics?
Defining the “performance envelope”
Let’s continue with our car ride analogy.
The development team took on a new task, requiring them to get to point C. If they had a choice between cars, would they take a 15-year-old preserved car or a brand new one?
When I pose this question to people, most of them say they would take the new car, but they are not sure why. With some additional prompting, they can typically explain that a new car has superior features: the ability to go from 0 to 100 MPH in a shorter time, a better brake system (AEBS) and autonomous features like lane control, adaptive cruise control (ACC) and more.
It all comes down to a better performance envelope, allowing the driver to care less about speed and stability and more about staying on task and on the route.
DORA metrics as the delivery performance envelope
For development teams, the DORA metrics encapsulate the delivery performance envelope of the service. The metrics let us know how fast the team can deploy and how stable the service is.
Knowing how well the service is in relation to these metrics tells us where to focus. If our service is fast and stable, we should focus on the feature at hand and on the roadmap. If it is not, we are probably wasting efforts on the delivery tasks like manual testing and manual deployment. In that case, we should focus on improving our delivery capabilities.
Other critical metrics
Let’s try now to answer the question of how DORA metrics are different from others like predictability and throughput.
It is like asking, “What is the difference between a) the time it takes the car to go from 0 to 100 mph and b) the average speed you drove the car last month?” While the former can be benchmarked and is comparable, the latter is important but doesn’t tell you how great the car is. It does tell you how well you are driving the car.
Team brochure
Anyone who has bought a car recently has seen a brochure, a standard document that outlines the specifics of the vehicle’s performance capabilities. They’re all there, neatly outlined and easy to understand.
Unfortunately, when it comes to development teams, there is no such brochure. Each service provided by the team must be measured and optimized continuously to ensure it delivers the expected outcome with high speed and stability.
The good thing is that if we see a metric with a gap, we don’t need to buy a new car. Instead, we simply need to upgrade our delivery abilities.
It is up to us.
Summary
Measuring collective team results across all four DORA metrics, as prescribed by DORA co-founders in their popular book “Accelerate”, enables the development team to clearly see its performance envelope. They can now understand whether they are already able to achieve their desires or still need to concentrate on areas that need improvement.
Exposing this envelope helps us narrow our focus and make more educated decisions as to how to sharpen team capabilities.
Are you measuring DORA metrics in your organization? Would you agree that these metrics are a performance envelope of your delivery capabilities?
