How to Quantify the ROI of Platform Engineering

Measuring the Impact

Platform Engineering is becoming the new “it” thing in software development, and it’s no wonder why. For starters, if we use an IDP (Internal Developer Platform) we can end up having fewer silos, better reusing components, and reaching an improved developer experience, such that they can self-serve and be more productive. However, adopting this new framework is not as easy as renaming our DevOps. It means the organization has to invest effort and money (especially if we are opting to build our IDP). So yes, all that is nice to have, and even quite important, but what’s the return on the investment? In this article, we are going to explore how to quantify the ROI of platform engineering.

Business drivers

First of all, we need to keep in mind what we are seeking in terms of business. For example, we want to know if we can translate developer autonomy into higher revenue or shorter time-to-market. In a recent talk at PlaformCon 2023, Dr Ralf Huuck succinctly presented the three main areas where a company can benefit from platform engineering:

• Predictability: with a platform in place, we gain insight into what we deliver, its cost, and its value. This translates easily into predictable business outcomes in terms of cost/revenue.
• Speed: an internal platform helps improve three different metrics that are related to speed:
• Time to market: how long it takes to deliver a solution to customers
• Time to react/fix: how consistent is our delivery
• Velocity: how fast are product features shipped

Better results in any (or preferably the three) of them benefit the agility of the company, meaning you can outperform competitors and be more flexible towards market demands.

• Compliance: everything around compliance and risk — may it be cybersecurity risk, quality issues, reputational risk, or just simply reporting obligations — can be unified and created much simpler in the course of action when using an IDP.

Now, let’s move on to how can we measure this so that we can translate advantages into quantitative improvements. How to quantify the ROI of platform engineering

This can be challenging due to the diverse range of factors involved:

1. Cost Savings: such as reduced infrastructure costs, minimized licensing fees, decreased development time, and savings from automation and process improvements.
2. Efficiency Improvements: for example, increased developer productivity, reduced time spent on environment setup and maintenance, faster deployment cycles, and improved resource utilization.
3. Time-to-Market: the time saved in development, testing, deployment, and release cycles can directly translate into increased revenue or market advantage.
4. Quality Enhancements: reduced defects, increased stability, improved performance, and enhanced user experience, which can lead to higher customer satisfaction, retention, and reduced support costs.
5. Developer Satisfaction: how platform engineering is positively impacting developers’ work experience, collaboration, and productivity.
6. Opportunity Cost: the potential risks, inefficiencies, and limitations of not investing in platform engineering. This can include increased development time, higher maintenance efforts, and missed business opportunities.
7. Business Impact: this includes metrics like increased revenue, improved customer acquisition or retention, accelerated innovation, and competitive advantage.

It’s important to note that some benefits of platform engineering may be challenging to quantify directly. In such cases, qualitative assessments, surveys, case studies, and anecdotal evidence can help demonstrate the value provided by platform engineering initiatives. Ultimately, a comprehensive ROI assessment should consider a combination of quantitative and qualitative measures to capture the full range of benefits and provide a holistic view of the value generated by platform engineering.

Here you have three well-known frameworks that can help you assess the ROI of platform engineering for your organization: DORA, SPACE, and McKinsey’s.

DORA

DORA stands for “DevOps Research & Assessment”. It is “the largest and longest running research program of its kind, that seeks to understand the capabilities that drive software delivery and operations performance”, as stated on DORA’s project homepage. It has become a standardized framework focused on the stability and velocity of development processes. The framework keeps them in context with each other, as a whole, rather than as independent variables, making the data more challenging to misinterpret or abuse. It is worth noting that it deliberately avoids the more controversial aspects of productivity measurements and individual performance metrics.

(Copyright https://devlake.apache.org/docs/DORA/#what-are-dora-metrics)

Within the velocity data cluster are two core metrics:

• Deployment Frequency: Number of successful deployments to production, how rapidly is your team releasing to users?
• Lead Time for Changes: How long does it take from commit to the code running in production?

Stability is composed of two core metrics:

• Median Time to Restore Service: How long does it take the team to properly recover from a failure once it is identified?
• Change Failure Rate: How often are your deployments causing a failure in production?

DORA is quite straightforward in the sense that it takes a handful of key metrics to get a reliable picture of how the team — or organization — is performing. To bring the model to life with your data, you can use DevLake, an open-source dev-data platform that collects, analyzes, and visualizes data from many developer tools like GitHub or Jenkins.

SPACE

The SPACE Framework is a way to measure the productivity of software engineering teams by using a holistic approach. It was first presented in the ACM journal Queue in 2021. Its name is an acronym that stands for the main factors taken into account in the evaluation:

• Satisfaction and well-being: how fulfilled, healthy, and happy developers feel at work, as a way to understand and even predict productivity.
• Performance: the outcome of the development process, in terms of quality and impact.
• Activity: the count of actions or outputs completed in the course of performing work.
• Communication and collaboration: how people and teams communicate and work together.
• Efficiency and flow: the ability to complete work or make progress on it with minimal interruptions or delays.

According to its authors, it is important to select just a few metrics that will represent each factor at an individual-, team- or group-, and system level. They acknowledge that most factors are difficult to measure and that individual metrics are inherently limited for properly describing them. In the article, they offer some examples (like the table below) and recommendations when attempting to implement the framework.

(Copyright https://queue.acm.org/detail.cfm?id=3454124)

McKinsey

Authors from McKinsey & Company have developed an approach to measuring software developer productivity that is meant to be easier to deploy using surveys or existing data (such as in backlog management tools). They looked to expand on the two sets of metrics already developed by the software industry– DORA and SPACE– with opportunity-focused metrics. There is an example of the framework in action in the image below.

(Copyright https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/yes-you-can-measure-software-developer-productivity)

The opportunity-focused productivity metrics provide understanding from several viewpoints that were not covered by the other two frameworks:

• Inner/outer loop time spent: Inner loop activities are directly related to creating the product: coding, building, and unit testing, while outer loop activities are other tasks developers must do to push their code to production: integration, integration testing, releasing, and deployment.
• Developer Velocity Index (DVI) benchmark: DVI is a survey that measures an enterprise’s technology, working practices, and organizational enablement and benchmarks them against its peers.
• Contribution analysis of individual developers to a team’s backlog: this insight enables team leaders to manage expectations, improve performance, identify opportunities for training, or rethink the role distribution within a team.
• Talent capability score: it is a summary of the individual knowledge, skills, and abilities in a given organization.

Conclusion

In this article, we have stopped for a second to analyze how companies can benefit from using an internal developer platform in terms of business goals. Then, we reviewed all the different factors that weigh in when assessing the ROI of an IDP. Thankfully, we can count on several frameworks that are widely used for this task to help us along. However, we need to keep in mind that they provide snapshots of how well or badly we are doing in a given moment. If we want to evaluate the gains, we have to measure through time and watch the trends.

Lastly, if you are not using a developer platform just yet but would like to get an estimate of how your company could benefit from it, don’t worry: fill out our ROI calculator to find out how much you can gain using Napptive.