How should we judge a product team?
We generally accept that it is best to build software through autonomous, empowered teams. However, how do we assess these teams? We empower these teams because it is impossible for senior management to understand the finer details of the problems and complex solutions that these teams are delivering. But if that is true, then how do we know that these teams are performing well, given we have already accepted we don’t know what they should do?
It is tempting to ask the team for all sorts of metrics on their performance (story points delivered, defect rate, experiments run), but often these mean very little without full context. And as we’ve just said, the management team won’t have the context. If we measure teams on these, then they may just game the system and optimise these metrics, which often doesn’t lead to more value for the company.
Equally, it is tempting to measure them on whether they deliver something on time, or whether they hit their OKRs. But this sort of assessment just leads to sandbagging of targets, and we end up rewarding teams that can negotiate easier deadlines. Once again, we’ve already accepted that their work is very complex, so it is not right to expect that they (or worse, others) can predict how long something should take.
The main problem here is that none of the above take into account how much value is delivered. That’s what really matters, so that’s what the team should be judged on.
So I suggest:
- Clarify what success looks like — what value they are expected to deliver
- Find a clear way of measuring that value
- Judge them in the short term on frequency of value delivery
- Judge them in the long term on total value delivered
What value are they aiming to deliver?
It’s not fair on the team if they don’t know what management sees as value. It might seem like the ultimate in empowerment if you let the team do whatever they want, but the organisation should have some sort of idea what value they want from the team. So give them some clear metrics to focus on. Let them know what they should maximise, and leave them to it.
What problem are they trying to solve? For who? How can we measure this value? This sounds obvious, but I see many teams that don’t have clarity around this.
Judge on frequency of delivery in the short term
Emphasise that you expect them to deliver value early and often. Why?
- It’s impossible to know in advance how much difference something will make
- Delivering early minimises risk
- Delivering early increases learning
- Delivering value early means value is delivered early!
Expecting every feature they build to strike gold is unrealistic. Equally, sometimes a feature might strike much more gold than expected. There is a fair bit of luck involved here. So judging them solely on total value delivered in the short term is often unfair.
And if in the short term they focus on frequency of delivery, then they will deliver something quickly, which will minimise risk and increase opportunities for learning. As a result, they are most likely to end up maximising total value delivered over the long run.
Now I’m not talking about how many releases a team does. A team may do many little releases that they don’t expect to improve the metric (this may also be a good thing). But the thing to measure here is how many times the team releases something that changes the metric, or at least they expect to.
Judge them on total value delivered in the long run
At the end of the day, what really matters is whether the team is delivering enough value to justify their costs. It’s unfair to judge them on this in the short term given the uncertainty of any product development, but over the long term this is all that matters. So judge them on that.
And if total value delivered is not high enough, then it might be because the team is underperforming. But it might also be because they are working on the wrong problem, or they aren’t resourced appropriately. This is what senior management should be debating.
Summary
In summary, don’t judge teams on hitting deadlines or delivering story points. In complex worlds these are not straightforward, and it ends up incentivising the wrong things. Whether the team is delivering value is all that matters, so make it clear what this value is, encourage them to deliver it frequently, and judge them on value delivered over the long run.
If you enjoyed reading please give some 👏 and/or leave a comment. For more stories like this check out my publication, The Agile Mindset.
