Measuring Software Teams

Todd Moses
Mar 9, 2019 · 6 min read

Since the beginning of software, managers have dreamed of having a development team operating at maximum efficiency. The problem is that software is a very risky business with seemingly large swings in productivity. Unlike manufacturing, the act of measuring progress is often difficult.

For example, do you know who your best developers are? Who is being over or underpaid? Who is not pulling their weight or who is causing more work for others on the team? Many managers assume they have these answers only later to discover they were wrong.

Instead of making assumptions, one must measure the team’s results. Often, measuring a team’s effectiveness provides surprising insights. For one, the most effective developer may not be the highest paid expert. Many discover that the mid-level developers are the most important contributors to the team. Sometimes, the least experienced members are out performing the senior developers.

To determine these facts, an effective measurement must be made. It can be done simply at first then improved upon incrementally over time or almost instantly through various software tools. Regardless of the method, the process of taking measurements results in positive changes to development processes.

Image for post
Image for post

Taking a Measure

A good way to begin the process is to ask every developer to take an assessment on their core technology. These are available online from free to expensive. After a few minutes of questions, a score is given. This score is not a complete picture but gives an indicator to mastery of the subject matter.

Often, generating a report of pull requests for a given time frame by developer will give a rough indication of productivity. Afterwards, pull a report from the project management and/or defect management software that includes bugs introduced by developer. This may be in the form of bugs by ticket or bugs by pull request. However, you will need it in a format that can be converted to bugs by developer.

Then, using a monthly salary report for each developer with the other metrics can give a cost per pull request. To do so, subtract the number of pull requests by the number of bugs then divide the salary by this number. Such a metric is useful to determine the most efficient coders during the previous month or sprint. For example, if our time frame is one month we have:

Number of Pull Requests: 30

Number of Bugs: 5

30–5 = 25

Monthly Salary: 80000

8000 / 25 = $320 (cost per pull request)

Often people are surprised by the results. Using the scores of the assessment, a correlation hopefully emerges between performance and subject matter knowledge. Just keep in mind that correlation does not necessarily mean causation. Many managers create a summary score from these metrics with the cost per pull request to provide a type of effectiveness score per developer. Something similar to a credit score for effectiveness at delivering code.

Image for post
Image for post

Caveats and Warnings

Regardless of the method used, the process of measurement will lack many parameters needed to have detailed explanations of productivity issues. For these, communication with direct reports and the development teams must occur. In addition, it is important to be open with everyone from the start on what you want to accomplish.

The method discussed above is not a complete measure. Instead, it should be viewed as a starting point. There are many commercial tools that provide a more complete view of both individual and team results. For example, a measure of how much time each developer takes from or gives to others on the team is not present or easy to measure. Some of the metrics desired may take some thought before they can actually be measured.

Consider that a very talented engineer tasked as Team Lead was found to have solid mastery of the tech stack being used. They were the highest paid by a large factor and responsible for mentoring young engineers. However, when scoring by the above method, this person was being reported as the most inefficient on the team.

In actuality, this Team Lead was discovered to be the most efficient due to helping others on the team complete more pull requests. For this and many other reasons, it is important not to take immediate action from this data alone. Instead, management must validate all findings.

How findings are reported along with what information about the measurement process is made public are important decisions. Developers may panic at having to take assessments while suspicion may surface if measurements are kept private. Instead, a process of sharing information gained along with the measures taken needs to be made available at some level. A balance of transparency and privacy must be maintained for the team to consider management’s efforts as positive.

Image for post
Image for post

Using the Information

Make each metric measured a starting point to a private conversation with each developer. Consider that having developers score low on an assessment may not mean they do not understand their job. Instead, it may just point to poor test takers. The same is true for pull requests.

For example, a young parent took a week off of work to care for a sick infant. The manager was not given this information by their direct report, the person’s immediate supervisor. Being an excellent developer, the person in question had scored very high on the assessment but had a low score on other areas of the measure due to their absence.

After discussing it with the direct report, the information of the personal time off kept the manager from stressing a person already under pressure from a sick child. Examples of this are nearly endless when it comes to managing large teams.

The goal should be improvement of a team by improving each person on that team. By implementing training, mentoring, and incremental improvements to processes, this data can result in more effective teams over a set time-frame. At the very least, it will show where the team needs to improve while making overall goals easier to formulate.

In sharing results, an awards system is useful. For example, having team awards for most productive, highest quality deliverable, etc. is much better than posting individual scores. After all, the purpose is to improve the team, not cause members to compete among themselves.

Image for post
Image for post

Conclusion

Knowing how many pull requests each developer makes along with the number of bugs discovered per defect is immensely helpful in determining a baseline of productivity. Using this information with other measures related to shipping completed code can give insight into how to make the team better while helping to establish profiles for each new hire.

Taking regular measures of teams and individuals makes goal setting more accurate. For example, if the average number of pull requests per month is 30 but three developers on the team are below that number, additional metrics will tell why. From there, a manager can work with those developers to improve.

Tracking initiatives through team measurement is the only way to discover the best methods for growth. Including educational investments, tool purchases, and bonus structures to name a few.

As always, thank you for reading. As you read, you probably thought of a friend or network to share it with. If so, please do so. To learn more about improving software teams, please visit https://toddmoses.com.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store