Developer productivity: How to measure it?
This blog was originally published in the Typo blog.
Developer productivity is notoriously difficult to measure — it is complex & nuanced with major implications for software development teams. A clear, defined, measurable developer productivity could provide engineering managers & leaders the ability to ship software faster & efficiently.
Your company is currently measuring your engineering team by output and not the processes. The CEO doesn’t care about commits, story points, or code written. They care about more features, happier customers & getting more without spending more money. So how can you quantify the software developer productivity to make the business/CEO care more? And how do you align with the business objectives?
What is developer productivity?
Developer productivity is a measure of the team’s ability (& not just the individual’s ability) to efficiently ship high-quality products that deliver business value based on the goals assigned.
How to measure developer productivity?
There are several myths about developer productivity. Understanding these myths uncovers a key takeaway about measuring productivity.
Productivity is not just about developer activity:
According to Nicole Forsgren in an interview for InfoQ who created the SPACE framework (we’ll be coming to it soon!)
One of the most common myths — and potentially most threatening to developer happiness — is the notion that productivity is all about developer activity, things like lines of code or a number of commits. More activity can appear for various reasons: working longer hours may signal developers having to “brute-force” work to overcome bad systems or poor planning to meet a predefined release schedule.
Measuring only the developer outputs is detrimental because of the lack of data points on the productivity gap at the developer level or company/surroundings.
Measure teams, not individuals:
Most organizations measure individual performance metrics like review time, code commits, size of commits, and code review frequency. This is not the right method to improve engineering efficiency.
The focus should be on the team, than just the individuals. Combine the above metric with the team-level parameters like team goal accomplishment, and collaboration and see how they are growing together.
Productivity cannot be reduced to a single dimension (or a metric):
Productivity is not just about one dimension like developer activity based on code commits, PR reviews, etc. It represents multiple factors ranging from SDLC metrics, to work distribution, and well-being at the team level as well. Co-relating cross-metrics gives an accurate picture of the team’s performance.
Why is it important?
A few of the reasons why developer productivity is important are stated below:
Business outcomes
Efficient software developers can ship high-quality code that delivers business value. Positive business outcomes accelerate the development process. This not only increases customer satisfaction but also results in faster product releases and new features.
Improves overall well-being
Developer productivity is not just important for business outcomes. It improves the physical and mental well-being of the developers as well. High productivity among developers leads to reduced stress and helps prevent burnout. Hence, it contributes to their positive mental and physical health.
Developer satisfaction
Developer productivity allows the development team to be highly satisfied with their work. This is because they can see the impact of their work and can accomplish more in less time, contributing to a positive Developer Experience. Besides this, they can focus better, set better goals, and be more effective at their jobs.
Improves quality of work
Productive developers are more efficient at work. They help them to strike the right balance between speed and quality of work. As a result, it reduces the need for rework and extensive bug fixes.
Improves creativity and problem-solving ability
Productive developers have more time to focus on creative problem-solving and innovation. They have better analytical and decision-making abilities. Hence, they can introduce new and innovative features and technologies that enhance their products or services.
Best practices for measuring developer productivity
Define clear goals:
Define clearly what constitutes productivity for developers. Make sure that it aligns well with their objectives, project requirements as well as organization goals.
Ensure coding standards and quality:
Monitor adherence to coding standards and quality to ensure code readability, consistency, and maintainability. Well-documented coding standards help in shared understanding among developers. Emphasizing code quality is crucial for producing reliable and efficient software systems.
Debugging efficiency:
Measure the time and effort taken by developers on debugging and troubleshooting. To make it less complicated, use debugging tools to identify issues and address them in the early stages.
Give honest feedback:
Use developer productivity tools and metrics (Such as the SPACE framework) to provide genuine feedback to developers. Make sure that the feedback is clear, specific, and actionable to let them know the blind spot and how they can improve it within a time frame.
What is SPACE Framework?
In the developer world, there is a state of hyper-productivity commonly referred to as “flow”. The framework uses several data points to gauge the productivity of the team & not just at the individual level. It combines together quantitative & qualitative aspects of the developer & the surroundings to give a holistic view of the software development process. SPACE is the acronym for satisfaction, performance, activity, communication, and efficiency. Each of these dimensions is key to understanding and measuring productivity, according to the researchers.
Satisfaction and Well-being:
The dimension of employee satisfaction and well-being is often evaluated through employee surveys, and it assesses whether team members are content, happy, and exhibiting healthy work practices. There is a strong connection between contentment, well-being, and productivity, and teams that are highly productive but dissatisfied are at risk of burning out if their well-being is not improved.
Although quantifying satisfaction and well-being is challenging, objective metrics can identify circumstances that may lead to dissatisfaction or burnout. By combining quantitative metrics with qualitative information and survey data, engineering leaders can gain a better understanding of team satisfaction. Work in Progress PRs is another useful measure, as a high number may suggest that developers are being forced to switch tasks too often, which can prevent them from entering a state of flow. To gain a more comprehensive view of team satisfaction, engineering leaders should examine both short-term changes and overall trends.
For instance, when a manager at typo noticed a sudden increase in the volume of PRs and reviews, she took it as an indication to investigate the workload of each developer. She discovered that engineers were working overtime and on weekends to launch new features and successfully advocated for extra time off to help them recharge and avoid burnout.
Performance:
The SPACE Framework originators recommend evaluating a developer’s performance based on their work outcome, using metrics like Defect Rate and Change Failure Rate. Every failure in production takes away time from developing new features and ultimately has a negative impact on customers.
The latter is particularly useful to measure the quality of code shipped to customers. In evaluating team performance, PR Throughput is a vital complement to quality metrics, counting the number of PRs merged over time to gauge productivity and progress.
Activity:
The Velocity framework includes activity metrics that provide insights into developer outputs, such as on-call participation, pull requests opened, the volume of code reviewed, or documents written, which are similar to older productivity measures.
However, the framework emphasizes that such activity metrics should not be viewed in isolation but should be considered in conjunction with other metrics and qualitative information. Among the useful activity metrics within the Velocity, framework are coding metrics, like Commits Per Day and Pushes Per Day, which enable leaders to assess the balance of work, workload expectations, and delivery capacity aligned with strategic goals. Additionally, Deployment Frequency is a helpful metric for measuring how frequently teams release new features or bug fixes to production.
Communication & Collaboration:
Teams that are highly transparent and communicative tend to be the most successful. This enables developers to have a clear understanding of their priorities, and how their work contributes to larger projects, and also facilitates knowledge sharing among team members.
Indicators that can be used to measure collaboration and communication may include the extent of code review coverage and the quality of documentation. One example of a collaborative practice that can enhance team performance is Pair Programming, which involves two developers working together to write code and can be attributed to both developers.
Evaluating the effectiveness of your team’s collaborative code review process can be accomplished through the use of metrics such as Review Speed, Time to first Review, and Review Coverage. These metrics offer insight into whether your team is prioritizing collaboration, allowing sufficient time for the review of pull requests, and providing meaningful feedback in comments. By analyzing these metrics, you can assess the overall health of your team’s code review process and identify areas for improvement in code quality.
Efficiency & Flow:
The concept of efficiency in the SPACE framework pertains to an individual’s ability to complete tasks quickly with minimal disruption, while team efficiency refers to the ability of a group to work effectively together. These are essential factors in reducing developer frustration. However, it’s important to note that excessive efficiency and flow can be detrimental to collaboration and feedback. Surveys can be used to determine perceived efficiency and flow, while objective measures of speed can be obtained through metrics.
In terms of efficiency and flow, Velocity considers Cycle Time and Mean Lead Time for Changes as important metrics. Cycle Time represents the time taken by a team to bring a product to market and a low Cycle Time implies faster delivery of new features and bug fixes to customers.
This metric offers an objective means of assessing process changes and evaluating the output of the team, which can indicate the team’s stability. Mean Lead Time for Changes, which measures the speed of code changes in DevOps, can be an indicator of team health and the ability to respond promptly to requests. Both metrics can provide insight into the effectiveness of a team’s culture and processes in managing high volumes of requests.
The SPACE creators also laid down the framework in action at three levels — individual, team & system as shown below –
