MEASURING SOFTWARE DEVELOPMENT EFFICIENCY & PRODUCTIVITY

11 min readJul 6, 2019

https://addontraining.com/wp-content/uploads/2016/10/sdlc.png, an example of an Agile Model

Abstract- Introduction: Software development is widely investigated and studied in Software Engineering literature. Yet, software development in itself needs to be constantly evolved to meet up with the demands of the new generation of power-users. This makes software development efficiency and productivity measurement a baseline for better software engineering.

Objectives: The main goal of this paper is to investigate how software development efficiency and productivity can be measured.

Method: This paper is based hugely on secondary data. This secondary data was obtained from large research projects, empirical studies and also research papers published online. The focus of the paper was to identify the factors that would affect software development efficiency and productivity from a managerial point of view.

1. BACKGROUND

Software development is a very human-intensive task and thus, it is very crucial to measure software development efficiency and productivity. Since software development efficiency or productivity cannot be directly tracked, calculated or measured, we need factors to identify them. These factors are later discussed in the paper.

Measurement also helps to identify scenarios when software development isn’t going in the right direction. These metrics thus can change the course of the software development and deliver a product without compromise in quality, requirement or time. According to Brook’s Law, from the book The Mythical Man-Month, “Adding manpower to a late software project makes it later.” Thus, the state of the software development should be identified before it’s too late.

This paper justifies the factors by answering the following research questions:

RQ: Why do we need to track software development efficiency and productivity? What is software development metrics? Who needs software development metrics? What are the advantages of software development tracking? Should teams merely be concerned with deadlines and time sheets, or should they focus on customer satisfaction to evaluate team performance? And most importantly, the factors to measure software development efficiency and productivity.

And also some other relevant questions.

These questions might become tough to answer in certain situations. This paper tends to clearly identify the factors that would be able to track the software development efficiency and also the productivity of it. Also, A study conducted back in 2008, estimated that 68% of surveyed companies were statistically unlikely to have a successful project due to poor project requirements. More specifically, they were likely to face the following issues:

Have a budget of 160% more than the original.
End up with 180% more of estimated time than expected.
Deliver less than 70% of the expected functionality.

Managers also often choose to track too little or too many parameters just for the sake of gathering some information. This poses a real challenge to team leaders and the companies responsible.

2. STUDY AND REVIEW

2.1 Efficiency v/s Productivity

A software development metric stands for a potential area where measurement can be effectively applied. To define metrics we first need to properly identify the difference between efficiency and productivity.

Productivity means the quantity of work done by an entity or individual, while, efficiency is the quantity of resources used per unit work done. These resources can be anything from time, computation power or even employees. Suppose, a team develops a two thousand lines of code in a day, which is good. This, it can be said that the team has good productivity; now, let us say that half of the members aren’t working at all. Thus, it can be said that the team has good productivity but bad efficiency. Thus, it can be easily concluded that:

Productivity = (Size of developed software)/(Software development effort)
Efficiency = (Size of developed software)/(Resources Used)

Since productivity and efficiency are both different concepts, we’ll separately define factors for both.

2.2 Software Metrics

Software metrics are quantitative measurement of a software product or project, which can help managers understand software quality, performance or the productivity and efficiency. These can broadly be classified into two types:

Size-related metrics
Function-related metrics

It becomes imperative to make measurement easy to collect or it will not be done. Make the software metrics work for the software development team such that the team can work better and is able to figure out mistakes and short comings. Measurement and analysis shouldn’t be too cumbersome or something that gets in the way of other goals or hampers smooth functioning. Software metrics should have several important characteristics; software development measurement would be useless, if metrics do not follow these characteristics. They are as follows:

Simple and computable.
Consistent and objective.
Use consistent units of measurement.
Independent of programming languages.
Easy to calibrate and adapt.
Easy and cost-effective.
Able to be validated for accuracy and reliability.
Relevant to the development of high-quality software products.

This is why software development platforms that automatically measure and track metrics are important. But software development teams and management run the risk of having too much data and not enough emphasis on the software metrics that help deliver useful software to customers.

2.2.1 Size-related metrics

These metrics are related to the size of the output of software development. Also, for better investigation size of any attribute of activity of software developed can be used. These metrics can be easily corresponded to statistical and numeric data. Few of the examples of size-related metrics are listed below:

1.Lines Of Code (LOC):

These are the number of lines in the source code, instruction path length, complexity, etc.

2.Developer productivity metrics:

These metrics help in understanding how much time and work developers are investing.

3.Agile Process metrics:

Such as lead time, cycle time and velocity.

These were a few examples of the size-related metrics, there are more examples like these. Thus, it can clearly be said that these are directly measurable.

2.2.2 Function-related metrics

These are related to the overall functionalities of the software under development. These represent the amount of useful functionality shipped during a set period of time. Few types of function-related metrics are given below:

1.Function points:

These are a way of expressing the functionality that is included in an executable software unit. A function point is not a single characteristic but is computed by combining several different measurements or estimates.

2.Application points:

These are alternative to function points. The advantage of application points over function points is that they are easier to estimate from a high-level software specification.

2.3 How do Software Metrics Lack Clarity?

The terms used to describe software development metrics often have multiple definitions and ways to count or measure characteristics. The dynamics and characteristics of the software development metric matter too. For example, lines of code (LOC) is a common measure of software development. But there are two ways to count each line of code:

One is to count each physical line that ends with a return. But it may include lines of “dead code” or comments.
To avoid these shortfalls, each logical statement could be considered a line of code.

Thus, a single software package could have two very different LOC number depending on which counting method is used. That makes it difficult to compare software simply by lines of code, which is why establishing a measurement method and consistent units of measurement to be used throughout the life of the project is very crucial.

2.4 Factors/Metrics to Measure Productivity

Since Lines of code, bug rates, etc., are not necessarily good indicators of how well or poorly your software development team is doing, we need better metrics to track them. These metrics can also be in the form of a report or graphical analysis. These metrics or factors can also sometimes be very complex when it comes to tracking productivity of agile teams.

Also, there are no ‘best’ factor or metric to track software development productivity. Team Leaders or Scrum Masters should always look for the factors that would be most suitable and most effective for their team. The factors and metrics that prove to be the most suitable in all cases are as follows:

Sprint Burndown
Release Burndown
Team Velocity
Control Chart
Cumulative Flow
Open Pull Requests

2.4.1 Sprint Burndown

Firstly, a burndown chart tracks the amount of software/requirement a team has developed across an iteration. These are mainly used to predict the team’s likelihood of completing the remaining work in the time remaining.

Figure 1: Sprint Burndown Chart (Courtesy of QuickScrum.com

Also, reading a sprint burndown chart is simple, the X-axis shows the days of sprint, Y-axis the effort (in this case: the hours remaining). Among the two lines, one line shows the ideal visualization of expected progress, other represents the current progress. In Figure 1, the requirement is met before expectation.

The sprint burndown chart makes the work of the team visible. It represents the rate at which work is completed and how much work is remaining to be done. What makes the chart effective is that it shows the team progress against the sprint goal, in terms of the work remaining.[1]

2.4.2 Release Burndown

This method is similar to Sprint Burndown, except for tracking days and hours, it tracks sprint sessions in the X-axis. This allows making burndown reports of longer periods of development consisting of multiple sprint sessions. Team Leaders are responsible to update release burndown data after every complete sprint session.

It is completely similar to Sprint Burndown but, bigger in scope-sprint instead of days.

2.4.3 Team Velocity

Team velocity is used to measure the “amount” of software developed during a sprint. The chart directly compares the amount of software developed to the amount of commitment of each sprint session.

The gray color depicts commitment and green depicts the actual work completed.

Whenever a change is made in the team, for example, a member is added or removed from the team, the team velocity chart will be affected.[2]

Figure 2: Team Velocity Chart (Courtesy of Daxx.com

2.4.4 Control Chart

Control Chart is a software development metric that shows the duration and how the process changes from start to completion.

A control chart always has a central line for the average, an upper line for the upper control limit and a lower line for the lower control limit. Team Leaders and Scrum Masters can compare the current data to these lines (which are obtained by historical data) to measure the productivity.

2.4.5 Cumulative Flow

Cumulative Flow aka Iteration Flow is a graph used to depict the team’s effort and project progress. It is an area graph hat depicts the quantity of work in a given state, time in queue, quantity in queue, and departure. IT visualizes the shift of work from one status to another as the team proceeds. Cumulative Flows will, at a glance, highlight bottlenecks and rough edges in your team’s process of completing their work.[3]

Figure 4: Cumulative Flow Area Graph (Courtesy of Kaban Tool)

2.3.6 Open Pull Requests

Pull request is the way a developer requests their team to review the changes or edits they have made and add those changes/edits to the repository. The number of open pull requests can be tracked directly using simple tools. Open pull requests can be measured daily, weekly or monthly basis. Measuring the number of open requests can allow team leaders or scrum masters to spot bottlenecks in requirement delivery process.

2.5 Factors/Metrics to Measure Efficiency

Efficiency, in software development, is simply the amount of software developed or requirement meant divided by the amount of resources used like time, effort, etc. Different teams and different engineers will have different efficiency rates. Efficiency is generally more when the same task is done by the same team over and over again.

Thus, the following metrics prove the most suitable to track software development efficiency:

Meeting Times
Lead Time
Code Churn
MTTR and MTBF
Impact

2.5.1 Meeting Times

Meeting times can surprisingly predict a lot about the efficiency of a team of software developers. If the team is extending their meeting times on a regular basis, then the team is going through a problem. At the end of every sprint session, team leaders or scrum masters should inspect the committed meeting time and the actual meeting time.

2.5.2 Lead Time

Lead time is the amount of time between the birth to the end of a process. It starts right back from the initial discussion or proposal of the idea or requirement to the development and delivery of it. It depends on both team quality and project complexity, both of which directly affect the project cost. [6]

2.5.3 Code Churn

Code Churn is the time the developers spend editing, adding, or deleting their own code. Basically, it measures the rate at which the code is evolving. Undoubtedly, code churn varies with project types, team quality and the course of the software life cycle.

Churn can vary drastically at different levels of software development. It primarily allows team leaders and scrum masters to measure the quality of the code being developed; since more the number of changes made in the code, the more evolved the code will get. [7]

2.5.4 MTTR and MTBF

MTTR stands for Mean Time to Repair and MTBF means Mean Time to Failure.

After successful software development and delivery, it is essential to fix issues like bugs and make efficient changes. These factors//metrics help in keeping track of loose ends even after software/requirement delivery.

MTTR is the average time required to fix a failed or repair a bug or issue. Similarly, MTBF is the records the failures due to design conditions.

MTTR = (total maintenance time)/(total number of repairs)

MTBF = (total operational time)/(total number of failures)

These metrics are useful because taking too long to repair can prove to be highly unpleasant and can impact the business. [5]

2.5.5 Impact

Impact is a measure of how the code is affected by the changes made in the code. It’s quite self-explanatory. The impact of a change set depends on a variety of factors such as amount of code, the seriousness of the code and the number of dependent functions and also files. A change in code that affect multiple files with addition and deletion at multiple locations will have more impact than a change made by adding a hundred lines of code in one file.

3. CONCLUSION

The job responsibilities of team leaders and scrum masters compel them to measure productivity and efficiency without any bias to boost requirement fulfillment and delivery. Thus, they need reliable, consistent, available and repeatable metrics/factors to measure both team and individual productivity and efficiency. There are even more software development metrics that can be chosen to track, but in the end, it all leads to delivering value and remaining fair in the competition.

References

http://isbsg.org/productivity/

https://efficientsoftwaremethodology.wordpress.com/esdm-home/advantages-of-esdm-methodology/

https://www.tasktop.com/blog/measure-productivity-software-development-team/

https://www.infopulse.com/blog/top-10-software-development-metrics-to-measure-productivity/

https://stackify.com/measuring-software-development-productivity/

https://setapp.com/lifestyle/productivity-vs-efficiency

https://www.sealights.io/software-development-metrics/top-5-software-metrics-to-manage-development-projects-effectively/

https://ifs.host.cs.st-andrews.ac.uk/Books/SE9/Web/Planning/productivity.html

https://en.wikiversity.org/wiki/Software_metrics_and_measurement

https://www.guru99.com/software-testing-metrics-complete-tutorial.html

https://www.daxx.com/blog/development-trends/top-software-development-metricss[2]