KPIs For Software Development Team Leads

Key performance indicators for the development team leads, and how to measure them in Azure DevOps.

Boris Donev
Jun 6 · 9 min read

The development team lead is obviously a very important link in the software development lifecycle. You’d expect them to not only be technically equipped, but also have man management skills.
While specifying and delivering technical solutions is great part of their job, making sure the rest of the team learns, improves and works towards the collective goal is of same importance. They should also be responsible and look into the project health.
As such an important part, a great care has to be taken for them — but also their performance needs to be somehow measured.

What will you measure depends on what do you expect from the team. Notice that we talk about what would you expect from the team — and not individually the team lead — because if the team performs well, then the team lead performs well too.

Let’s see what are some key performance indicators that could point us to potential problems in the team.

Quality of work

The number, or lack thereof, incidents, critical bugs, regressions etc. could mean that the team is doing quality work. There is nothing that impacts clients or users in an extremely bad way.

  • Incidents
    Note that incidents are not bugs, but rather something that happened that degraded the system’s performance or put its security at risk. An example of an incident we had is some beta access keys were made public on GitHub unintentionally.
    Fingers should not be pointed here, but rather you should talk about how to automate or modify the process so the incidents are less likely to be repeated.
  • System downtime
    The system can go down for whatever reason, not specifically because the team made a mistake. Nevertheless, it impacts the users, so if there was any downtime, it is reasonable to look into it and why did it happen.
    This can easily be measured with variety of tools out there. We use Application Insights' Availability Test, in combination with the Application Insights widget in the dashboard.
Preview of Application Insights’ Availability Test
How the widget looks in the dashboard
  • Critical bugs
    Every application has to deal with critical bugs that need to be solved immediately, and are impacting users.
    You can measure them by creating a query in Azure DevOps. Take all bugs that were closed or resolved in the last 3 months, that either have Critical or High severity. If you follow the git flow model, you can also search for ones that are tagged as a “Hotfix”. Then, you can add the query results in the dashboard.
You can use the query results widget for this
And this is how this query is defined in Azure DevOps
  • Regressions
    Regressions imply that double, or maybe triple the time is spent on the same bug, just because it was not investigated well on the beginning, or the root cause was not found initially. The applied fix was actually a work around, which just adds technical debt and is not the way to fix bugs.
    In Azure DevOps, you can create a very similar query to the above, just find the bugs that are tagged as “Regression”. This is a bit error prone, because the team might not tag all regressions correctly, so you have to take that into account.
You can choose to use query tile for this one, and add some visual clues
  • Unexpected events
    While most of the accidents that happened around the project would be covered with the previous points, you could let the team lead talk about other unexpected events that put the users or stakeholders in awkward positions, or put pressure on the development team.
  • Bugs monitoring
    Bugs are bound to happen in any project. How the development team reacts to them is what matters. If the team finds bugs out by manual testing or stakeholder emails, you might need to react because the team does not actively monitor issues. However, if the team reacts on automatic alerts by the monitoring system, then there’s not a lot more they can do, because they would have found and probably fixed the bug before the users even know.
    If you use Application Insights, you can monitor bugs by adding their widget into the dashboard.
How the dashboard widget looks like
  • Performance monitoring
    The system can be the best one out there, but if it is slow, the users won’t stay long. It is very important to check that the team monitors the performance as well.
    If you use Application Insights, apart from tracking bugs, measuring performance is one of the things that come out of the box. You can add more widgets to the dashboard so you can have the average response time, and maybe the number of requests if that makes sense in your scenario. In reality you should measure the 95th or 99th percentile, but the widget can give you an idea at least.

Teamwork

Making sure the team is like a well oiled machine and works without blockers is one of the main jobs of the team lead.

  • Sprint goals
    Planning work upfront, and letting all the developers know what their job is in the next sprint could mean that work is well organized and everyone knows what to do.
    You can use the sprint goal extension, or tag stories with “Sprint goal” tag, so the team knows what to focus on. Then, you can check if sprint goal is set for the last period by either checking the extension or doing a similar query like above, just searching for “Sprint goal” tag.
  • Cycle time
    This shows how much time a user story needs to go from active up until completed state. It’s an important metric because it tells if the team creates too big user stories, have bottlenecks in peer review, has unmaintainable code or maybe some generally slow processes. The cycle time should be smaller than 1 sprint, and should be ideally (according to me) ~30% of the sprint length.
The cycle time widget is perfect for this
  • Active & completed pull requests by the team
    Looking at the state of the team’s active and completed pull requests in Azure DevOps can also uncover issues.
    Are there any old pull requests? That can be danger of losing that code, as merging it becomes more expensive the more time has passed. Requirements could be changed, it should be tested again, could have many conflicts and so on.
    Are there no comments on the pull requests? It could mean that the developers are not taking the peer review process seriously enough. Bugs could be introduced, new members of the team learn at slower pace, different parts of the project could have different patterns (thus more difficult to read and debug which will again result in more bugs or more time to resolve bugs).
    Are there too many active pull requests? It could mean the team are not actively peer reviewing, which brings most of the issues listed above.
    Any bypassed pull requests? It could mean the team skips the automatic policies, which could bring the project in a state where it could not be built, and it will take more time to fix that after some time has passed. You can get a rough idea of all these questions just by glancing on the pull requests sub menu.

Investment profile

The client invests their resources in the development team, so they will get valuable new features on the product. However, the client is also responsible for many emails with one-time tasks or reports, requests for estimation or ad-hoc meetings for new features. Even though they think this is a 1 minute job and won’t take anything out of the sprint, in reality, it defocuses the developers and has a greater impact.
Another impactful thing on the client investment is the number of bugs — knowing this information and the support tickets as a raw number in a period of time will help you devise communication tactics with the client.

  • Story to bug ratio
    If the team mostly works on bugs, that means the investment by the client does not really pay off. Obviously there will be bugs, as mentioned before, but they should be kept within a meaningful ratio.
    Ideally, bugs should not be more than 20–30% of the number of user stories.
  • Support
    Usually the designers, sales or other stakeholders would have “simple” questions for the development team, or request a “simple one-time” reports from the database. While these tasks take no more than 15 minutes, it defocuses the developer and it will take much more time for them to focus again on the problem. It is understandable for non-developers to not understand this, but every such task should be tagged with “Support” tag, so you can come up with a report to the stakeholders later down the line. If you present facts, that you had 30 support tickets in the last 3 months, and for that time you could have implemented 3–4 more larger user stories, the stakeholders will think twice before asking for simple things again.
You can use the query tile widget for this — more specifically 3 of them
And the query for the user stories

Measuring this in Azure DevOps would require you to create 3 fairly simple queries. Just search for all created user stories in one, and bugs in the other query within the last 3 months. For the support, you need to query for any work item tagged as “Support”.

Other

Apart from measurable things, you should also discuss an array of open ended topics with the team lead, which can give you a great insight into the way they operate and manage the team.

Talking about new features that they developed recently and the technical decisions behind them might give you an insight on their technical readiness. It’s fine if you even deep dive into part of a code or a pull request. Refactoring taking place, especially for a second time on the same part of the code could mean that features are not planned well in the first place. Initiatives, or lack of them, can also uncover problems in motivation or interest in the project. Talking about how the team lead motivates the rest of the team can show you a great deal of their character and way of managing team and expectations.

The team lead should also greatly comply with the company policies. Do you have guilds and chapters, or other way of encouraging shared knowledge? Team leads should be main contributors there.

The team lead should also have a clear image of where they need improvement, and how to get there.

Obviously one team lead cannot be master of all these points, but should be above average on most of them.

Conclusion

Since most of the KPIs directly refer to the team lead’s work, make sure the lead is well aware that this is not a judging meeting and you are not trying to be smart, but you all are trying to spot if there are problems and issues in the team, and talk about how to fix them so the team — and the team lead — become better.

By covering and making sure incidents, downtime and bugs are reduced, you mitigate the issues that directly impact end users and the client. Looking at the sprint goals and cycle time, you ensure up to some point that the team and it’s processes are efficient enough. You can also manage clients by measuring support tickets and letting them know of the extra workload they produce. And the key in measuring all of these is enriching the work items with data when they are created. Marking them as support or regression, or setting the severity can give you so much information, and help you make informed decisions on the next steps.

Nerd For Tech

From Confusion to Clarification

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.

Boris Donev

Written by

As a technical team lead of several startup projects, I've come across different issues which I'll try to document and maybe help someone else.

Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.