Making Sense of Board Metrics
Here at carwow, we’re big fans of data. Important decisions in any area of the business should never be made based solely on assumptions, or finger-in-the-air hunches, but instead supported by cold, hard, evidence.
We think we’re pretty good at understanding and analysing the results of our work. But what about the process of work itself? All our product teams use Kanbanize as a means of visualising our workflow. Like most similar products (e.g. Jira, Pivotal, Asana), it comes with a powerful range of metrics to visualise exactly how work is flowing across your board… that we didn’t understand 😖
So Rory, one of our Senior Engineering Managers, decided we needed to work on understanding this better. He started a book club, and selected the Team Guide to Metrics for Business Decisions. The book lent itself very well to the weekly format; six self-contained chapters that each answer a different question about our work, and end with actionable suggestions that you can put in place immediately.
I won’t go into everything we learned during the club’s duration, but here are a few of the subjects that resonated with me:
Stop estimating, start forecasting
None of us like to admit it, but we humans are terrible at estimating how long a piece of work will take to complete. We want people to be happy with us more than we want to be accurate. Estimating correctly requires actually doing a large amount of the work to reduce unknowns. Stakeholders want a concrete deadline to work to.
It’s a difficult trap to get out of. So what do we do instead? Well, it turns out that all the work we’ve done previously can help us predict our future output. You don’t even need a lot of data to get started!
In Kanbanize, this is super easy. Assuming you’ve created an initiative, and linked some cards to it, a Monte Carlo simulation showing varying levels of confidence is just a couple of clicks away:
You don’t even need to have the work completely broken down. If there are still unknowns, you can add additional filler cards.
To this, you might say: “But the size of our cards is variable! How can this possibly be accurate?”
Stop scoring, start breaking down work
Scoring cards is another form of estimation, and can be just as inaccurate. While it’s on a smaller scale, mistakes compound, and cause projects to overrun.
So what should you do instead? Focus your energy on breaking work down into the smallest possible chunks. Once your team gets good at doing this in a reliable and reproducible way, you start to notice the benefits. Your average cycle time — the time it takes to complete a card — decreases. Throughput — the number of cards you’re completing — increases. Completing lots of smaller pieces of work at a constant rate has a huge effect on morale and motivation; the feeling of Getting Things Done.
And when you narrow the range of variability in cycle time, it makes your forecasting incredibly accurate, and your stakeholders love you again.
Looking at the example above, you can see the outliers decreasing over time, to the point where 95% of cards are finished in 6 days or fewer. Any outliers are a great chance to retrospect and discuss with your team how you can further improve your processes or card-dissection skills and have a lower degree in variability. Once you get to this point, you might change your definition of an outlier and look at anything above the 85% line for instance, and reduce cycle time even further.
Things are going well! But there’s still one more piece of the puzzle.
Stop starting, start finishing
There’s a temptation to think that engineers working in parallel will get more done than if they work together on fewer cards. But is that really the case?
Little’s Law gives an interesting viewpoint on how to make cards move faster across your board. It originates from queuing theory, and states:
Essentially, this means that: if you want to shorten the lead time (AKA cycle time, AKA the time spent from start→finish), you have two options:
- Increase throughput
This could mean removing blockers, improving processes, or asking your team to work harder. I wouldn’t recommend the latter option. - Reduce WIP
Encourage your team to pair on tasks, swarm on blockers, and focus on finishing rather than starting work.
Doing less to do more may sound counterintuitive, but looking at a cumulative flow diagram (CFD) can help visualise the three factors:
As the green band rises, we’re starting work. As the blue band rises to the same level on the Y axis, we’re finishing it.
The highlighted section is a great example of what not to do: the thick green band means multiple pieces of work are picked up, but the blue section isn’t expanding at all. Nothing is getting finished. There’s no flow.
Contrast that to other areas of the chart, where the green band remains thin and the blue band rises more sharply. The ideal CFD would be as close as possible to a pair of parallel blue and green lines, rising in tandem, but of course in reality that’s unlikely to happen. It’s good to have ambitions, though!
Increased flow isn’t the only benefit of reducing WIP, either. To name just a few of the others:
- Clearing blockages
Having more people focussed on a single task makes blockers more visible, and makes it easier to clear them. Teams can swarm on blockers in order to understand the problem and find the solution. It also makes bottlenecks in our processes more clear, allowing us to more easily iterate on them. - More visible progress and momentum
As the work flows across the board more smoothly and predictably, there’s less of a feeling amongst the team of work being a slog. - Reduce knowledge silos
There’s a tendency with parallelising work that people self-select into picking up similar tasks, resulting in one person holding all the knowledge in one area of the codebase. Having multiple people work in the same area shares knowledge, meaning that if — for instance — a person is ill, or on holiday, we aren’t blocked while other people get up to speed - Decide as a team on what’s the most valuable thing to do
Working on everything at once shows that we don’t care if the thing we’re working on is valuable. Finishing each thing faster tightens our cadence and allows us to decide more frequently whether the work we’re doing holds value.
Does it work?
All in all, Team Guide to Metrics for Business Decisions has been hugely useful for me in terms of not only understanding how to view all the metrics available to us, but how to work with the team to improve our processes.
Every piece of the puzzle is important, but they don’t all need to be put in place straight away. Regular, small improvements go a long way towards building a happy, high-performing team. And the great thing is, you can see your progress right there in the stats and use them to either celebrate success, or retrospect on any blips that might help improve your process further. Kaizen!
