Measure and Visualize Performance through a Scoreboard Index
We are going through an agile transformation in the past 2 years. 8 month ago we started to measure and visualize our agile transformation through a scoreboard.
We have defined 5–7 qualitative objectives that our executives cared about like “improve products quality”, “improve customer value delivery”, etc. For each of those qualitative objectives we have defined a few quantitative objectives like “less defects coming from external testing teams”, “less defects being reported by customers”, “shift into personas oriented backlog”, “deliver value frequently by sprint”, etc. We ended up with about 25 quantitative objectives overall.
We have defined for all objectives four levels on the scale of zero to three, where each level is represented by a color: 0 — red, 1 — orange, 2 — yellow, 3 — green. We have built a reference guide that defines those levels for each objective (couple of sentences of clarification). Each time a team moved beyond a level, we changed the color on the scoreboard. We considered the red and orange levels to be something we are not happy with, and the yellow and green ones something we are excited from.
We took all the teams we have and the objectives we have, and we have visualized that on a board, creating an heat-map. The columns are the quantitative objectives, the rows are the teams. Below are 2 examples of heat-maps we have built in Israel and in India.
Those heat-maps are updated on weekly basis by the agile guilds (15–30 minutes weekly is enough). Our agile guilds meet on weekly basis and using those heat-maps as a working tool. The agile guilds select one or more quantitative objectives from the board and work on that in the coming week. Each week we assess the progress and update the board accordingly. It is important to note that the agile guilds are self-organized and own those objectives. The guild chooses which quantitative objective to tackle and in what order. Moreover, the quantitative objectives are being changed from time to time. Sometimes we eliminate some, sometimes we add some. We try to do that on a quarterly basis.
We have an online version of those boards and we take snapshots on regular basis, so we can generate an excel graph for the team to see their progress. Once a quarter we plan to celebrate the success of the highest score teams by announcing them publicly in the company. We also celebrate the success of the teams that made the biggest progress since last quarter, so even new teams can be part of this celebration.
For more information on scoreboards and metrics, please read Management 3.0 Scoreboard Index.