Mastering Backlog Refinement

Most teams initially struggle with backlog refinement. Recently I realized why.

Product Backlog refinement is one of the most elusive activities in Scrum. A seemingly “magic” process that converts a wish into something that can be worked on by the development team. The official Scrum guide devotes only the following paragraph to Product Backlog refinement:

Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog. This is an ongoing process in which the Product Owner and the Development Team collaborate on the details of Product Backlog items. During Product Backlog refinement, items are reviewed and revised. The Scrum Team decides how and when refinement is done. Refinement usually consumes no more than 10% of the capacity of the Development Team. However, Product Backlog items can be updated at any time by the Product Owner or at the Product Owner’s discretion.

Long story short: you can do refinement however you want, as long as it does not take more than 10% of your time. But even if it does, that is just fine as well. With only this abstract paragraph for guidance, it is not surprising many teams have trouble doing proper Product Backlog refinement.

Effective refinement is essential to make Scrum work. Without proper refinement estimates are off and developers are more likely to make wrong architectural decisions. The team will deliver less value because they do not understand why their work matters. The velocity of the team will not reach a stable and predictable level.

Taming The Three Types of Uncertainty

Refinement is all about managing uncertainty and creating common understanding. This way the team will be able to estimate with confidence and help deliver business value.

There are three essential topics you should cover during refinement to help ensure a solid foundation for building the right product and building it right:

  1. Purpose uncertainty: why do we want to build this?
  2. End uncertainty: what do we need to build?
  3. Means uncertainty: how can we build it?

Purpose Uncertainty: Why Do We Want To Build This?

The first thing a PO should always discuss in refinement is why the feature we want to build matters. What is the value we expect this feature to deliver? What do we want to achieve?

Purpose clarity empowers your team to provide feedback. Maybe there is a much better way to achieve the same thing. Maybe your team will tell you that we should not build it at all because it will never deliver the expected value.

A clear why provides the team with the freedom to make decisions when they are coding. Your team will be empowered to both validate and verify the solution they come up with.

“Building the product right, but forgetting to ensure you are building the right product is a deadly sin.”

Failing to provide clarity on the purpose means your team is not able to help justify decisions from a business perspective. Building the product right, but forgetting to ensure you are building the right product is a deadly sin. You will exert a lot of effort without realising any business value. It is important to always let your team know why what they are doing matters. So your team can always keep the purpose in mind.

End Uncertainty: What Do We Need To Build?

The Development Team need to have a clear understanding what to build. During a refinement session the Product Owner joins the team to elaborate the feature. Together they clarify what to build, depending on what is technically feasible.

“The Product Owner attempts to reach the ‘Goldilocks zone’ of describing requirements: clear enough to prevent a gazillion questions, yet not so intricately described to silence discussion.”

When discussing a feature for the first time the requirements should be sufficient to provoke discussion. Requirements should not be described with such detail the scope is no longer negotiable. This prevents your meeting from turning into a Sweet Sixteen refinement session.

The Product Owner attempts to reach the ‘Goldilocks zone’ of describing requirements: clear enough to prevent a gazillion questions, yet not so intricately described to silence discussion. Refining the issues together creates common understanding. It makes sure every team member knows what to do.

If during refinement the team starts asking a lot of questions then this is a clear sign that the issue was poorly described. The right course of action is to write down all the questions and adjust the description based on the feedback. The Scrum team should discuss the feature again at a future refinement session.

If no questions are raised this can be both good and bad. If the feature is complicated, then you should check if everybody really understands what to build. Sometimes the team just says it is clear, because they see so many requirements that they feel overwhelmed.

Means Uncertainty: How Can We Build It?

When the whole team is on the same page on what to build and why, they still need to have a rough idea how to build it. If the team has no idea how to build it, then the work cannot be accurately estimated. There is also an increased risk of building a solution that is not feasible from technical perspective. It is important the team is confident being able complete the feature.

If the team has no idea how to build a feature, then plan a spike. A spike is a time-boxed investigation to figure something out: in this case how to develop a feature. One of the team members receives X hours to investigate how to build the feature. The next refinement session the team member explains how it can be built. Then the team knows what they need to do and can indicate if they are now able to estimate.

“The end uncertainty should be sufficiently low that the team can estimate with enough accuracy to get a stable velocity.”

When starting with Scrum many teams will really want to nail down all details and try to reduce the means uncertainty too much. Just like with end uncertainty, it is important to hit the ‘Goldilocks zone’. The end uncertainty should be sufficiently low that the team can estimate with enough accuracy to get a stable velocity. It should not be such a detailed plan that discussion of the issue is no longer necessary at Sprint Planning.

Concerning Epics

I have consciously not discussed how you should handle epics during refinement. Epics are issues which are too large for your team to work on and need to be broken down in smaller parts. You can perform all steps in this article on an epic. The final step, once all three topics have been addressed, should be how you split the epic into chunks of work for your developers. I will leave this for a future article, as there are many different approaches to consider.

Embracing Uncertainty And Creating Common Understanding

Product Backlog refinement is all about managing uncertainty and creating common understanding. Viewing refinement through the lens of taming the three types of uncertainty helps to make sure we have covered all bases adequately. When discussing means- and end uncertainty a fine balance should be struck. You do not want to end up in analysis paralysis or a Sweet Sixteen refinement session. You want to prevent doing work which you should do during the Sprint or Sprint Planning.

We all want to do work that matters and feel empowered to deliver more value to our customers. Bring the team in on your goals and they will help to achieve them in ways you did not expect.

Further Reading

  1. https://www.mountaingoatsoftware.com/articles/the-certainty-of-uncertainty
  2. https://www.mountaingoatsoftware.com/blog/product-backlog-refinement-grooming