Data Led Retrospectives with #Softwaredevelopment teams

With thanks to

Data and analytics are probably the most important thing to help you discover what the user does and how to improve your service. But it could also be used to help discover how the organisation works and problems within it.

Currently software development teams are asked to estimate how long things might take or when something will be done. This often fails, as it is an educated guess, so the alternative could be to develop and study analytics on how long things take for software development squads to get something done. This takes the estimation pressure off the group and makes the “when will it be done” question more scientific.

I’ve been lucky enough to work in many places where we built algorithms and simulations to try and estimate how long things take. This had varying results on both occasions but we improved over time.

The one area where data can really help is retrospectives. Data led retrospectives are often over looked. But how do you do them?

Visualise your process

If you haven’t done Kanban before please find David Anderson’s blog on the principles here and read the David’s Blue book Kanban.

Step one — Visualise your current process on a Kanban board. Ensure that you don’t put in the process you ideally want but visualise the process you have.

Step Two — Put in place WIP (Work in Progress) limits so you can regulate the amount of work in the system and ensure average arrival rates are the same as departure rates. Ideally you want to make sure the average age of WIP is never decreasing or increasing and is the same at the beginning and the end.

Step Three — Make sure all work is of a consistent size. For example all the features remain the same size, while the bugs are broken down into a suitable size

Step Four — Decide on your lead time. Where should it start and end? Should you for example start measuring when you select work or when you actually start development work.

Step Five — Make sure the work gets blocked in the same column. Never move the work back or off the board. You can find more details on this in my blog post here.

The Measurement

It is generally accepted that knowledge work such as Software development will always have high variation. The chart above is an example for feature development work done by Team C. In this example they did not keep the features at a regular size. Some small items were done within 2 -3 days while others took more than 11. The idea is to try and make work a regular size so you can better predict the time it takes to get things done.

It was Walter A Shewart and W. Edwards Deming that looked at variation within a process. They defined two types of variation — Common (Natural) and Special cause variation.

Common Cause Variation is something that we can’t do anything about. It has been described by Walter A Shewart as chance-cause. The Western Electrical Company used the term natural patter. It has also been described as background noise or something that just happens.

Special Cause Variation is down to things that have an assignable cause or a signal and is an unnatural pattern. We’re looking to identify both common and special cause variation but we can only fix special cause variation.

Collecting data before the retrospective starts

With data led retrospectives before you talk to the teams about the data you need to collect the evidence. Why is a feature taking 40 days worth of development?

The example you can see here that the feature moves back and forth and spends a lot of time waiting. You may know the reason for this but you should not impose your hypothesis on the team. You need to let the team discuss and find out the reason for the time it took. Then try and put in place improvements to make sure this never happens again.

Pre Retrospective

Firstly think about who should hold the retrospective. Should it be someone from outside the team? The advantage of doing this is that they won’t know the background of how things work within the team and they will ask the right questions.

Secondly think about how you should visualise the data. Should you use Cumulative flow diagrams, tables and Control charts?

Once you have everything prepared you then need to book a couple of hours to discuss the data with the team.

The Retrospective

Display the data starting with the Cycle Time. Don’t put in any opinions just talk about the data and what it shows. Stick to the facts.

Then review the individual feature/story data you have prepared. Ask the team to talk you through the feature. You can ask questions like:

  • What is the story behind the way it progressed through the development process?
  • How did the development go?
  • How did it do in code review, testing etc?
  • What can be improved or changed to make things better?

What you’re trying to get is the reason behind why the feature progressed like it did. Also to discuss improvements that could be made.

At the end of the retrospective you should try and summarise by getting the team to vote on one improvement or change that can be made in time for the next retrospective to study the results.

Data led retrospectives can reveal issues which you can ignore or try and fix to improve delivery times. There are very few occasions where the team should ignore the data. Ultimately you’re looking to improve things and data led retrospectives can help you open up the box and look inside so you can make things better for the team and organisation.


David Shrimpton has given conference talks and written articles on Agile and Lean and loves helping individuals through coaching. Currently he is the founder and Company Director for DPJS Enterprises.

If you’d like me to talk at your company or conference please feel free to contact me on Linkedin.

Like what you read? Give David Shrimpton a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.