Software Delivery Metrics
Part of The PIRATE Way — Stories about scaling up engineering teams.
Did you know that for many tech startups, the largest budget allocation is often reserved for engineering professionals? How do you ensure that every dollar spent translates into optimal outcomes? Delving into software delivery metrics can offer insights.
The Pioneers of DevOps Metrics: DORA
Within the vast terrain of DevOps research, DORA (DevOps Research and Assessment) stands out, sculpted by the insights of Dr. Nicole Forsgren, Jez Humble, and Gene Kim. Their collaboration led to the game-changing book, “Accelerate: The Science of Lean Software and DevOps”, unearthing the practices and metrics that propel top-tier tech organizations to their zenith.
Google Cloud’s acquisition of DORA in 2018 was more than a merger; it was a testament to the groundbreaking impact of DORA’s research. Nestling DORA’s findings within its ecosystem, Google reinforced its mission: aiding organizations in achieving software delivery excellence.
Are you curious about the latest in DevOps metrics under Google’s umbrella? Dive into the most recent report here.
Highlighting the metrics identified by DORA:
- Deployment Frequency (DF): The frequency of production pushes.
- Lead Time to Changes (LTTC): The duration from code commitment to its live run in production.
- Change Failure Rate (CFR): A measure of deployment issues — incidents, rollbacks, and failures.
- Mean Time To Recovery (MTTR): Time required to rectify a service post-incident.
These metrics provide a blueprint, helping organizations gauge their current state and the journey toward an ‘Elite’ working mode. For visual clarity, here’s a schema.
In 2021, DORA further expanded its metrics palette by adding:
- Operational Performance: Focused on reliability, encompassing availability, latency, performance, and scalability metrics.
More recently, in 2021, the DORA team incorporated a fifth metric:
- Operational performance, based on reliability, specifically the ability to meet or exceed their reliability targets (availability, latency, performance, and scalability)
Unraveling CI/CD Metrics
For those new to the DevOps realm, understanding CI and CD is pivotal. Continuous Integration (CI) and Continuous Delivery/Deployment (CD) lay the foundation for modern software engineering, sharpening the software delivery process.
While CI emphasizes integrating code changes frequently and detecting integration errors, CD ensures the software’s consistent readiness for release. The advanced version, Continuous Deployment, automates the entire release process, granting the changes clear the automated tests.
As you mature in adopting CI/CD, especially with team expansion, it’s vital to monitor:
- Build Time: Time consumed for system build. Excessive durations can drain CI credits and delay contributions.
- Testing Time: A significant chunk of the CI pipeline.
- Waiting Time: Arguably the most agonizing metric, it represents the average time a contribution stalls before CI system initiation. Elevated values might hint at competing PRs for the same repository or tooling capacity issues. Reducing waiting time minimizes cognitive load, ensuring smoother transitions for developers.
Another pivotal metric is the Build Ratio, reflecting the success-failure balance of build processes. Often, a protracted build time aligns with a low build ratio. If CI processes outperform local environments, developers might lean heavily on CI for testing, potentially compromising the system.
In conclusion, software delivery metrics are indispensable for any tech organization aiming for peak performance. They offer not just a snapshot of the present but insights for future growth and efficiency. Stay tuned as we delve deeper into these metrics in our subsequent articles.
Remember: This is a blog post from the series “The PIRATE Way”.
