A deeper understanding on…
The Product Backlog
Road to PSM III — Episode 13
So, after 12 episodes it’s about damn time I started writing about the more concrete elements in play.
“The Product Backlog is an ordered list of everything that is known to be needed in the product.” — The Scrum Guide
The Product Backlog makes unrealized (potential) value transparent. It’s value to be. The Product Backlog is the domain of The Product Owner. The Product Owner, being the Value Maximizer, is thus responsible for maximizing the value of the product resulting from the work the Development Team does. That is not to say the Product Owner is the only one to utilise the Product Backlog. In fact the Scrum Team is not the only consumer of the Product Backlog. It needs to be transparent to the level needed in order to maximise value.
It is not unfitting to use the ‘tip of the iceberg’ metaphor cliché. The Product Backlog should do the job of making sure only the most valuable stuff, that which the team needs to focus on, is visible and transparent. The Product Owner spends a fair amount of time submerged, in turbid waters, discovering valuable deep sea mysteries and bringing them to the surface.
I’ve personally come to value the Product Backlog as it is able to provide a Development Team with structured and transparent input to work from, within complex or chaotic environments. That said, in many cases, when the Product Backlog isn’t properly managed/refined, it may end up becoming a messy hodgepodge mirroring the organisation’s state. A Product Backlog may reveal a lot about the organisation, the personality of the Product Owner, and possibly even hint at potential nepotism. First time inspections of Product Backlogs, which (is my experience) generally reside in spreadsheets or digital tools like Jira, are generally unnerving.
Product Backlog items
Now the content of the Product Backlog is organised in Product Backlog items (PBI). Many refer to these as User Stories, but they aren’t exactly the same thing. Although User Stories (originating from Extreme Programming), are commonly used by Scrum Teams, they are not part of the Scrum Framework. A PBI may contain User Stories or anything that is known to be needed. But here is what the Scrum Guide tells what attributes they should in fact contain.
Product Backlog items have the attributes of a description, order, estimate, and value.
- Description
- Order
- Estimate
- Value
“More precise estimates are made based on the greater clarity and increased detail; the lower the order, the less detail.” — The Scrum Guide.
These may relate to features, functions, requirements, enhancements and fixes for future releases. So to answer one of the frequently asked questions related to the Product Backlog… yes they should contain fixes too! generally accompanied with its original Bug/Defect Report.
Defects/fixes
Attributes of a good report are:
- One defect per report
- A clear, instructive title, with a professional, non-sentimental description.
- A reference to the original specification
- Expected vs observed results
- Steps to reproduce
- When possible, recordings of the defect in action.
Test descriptions
“Product Backlog items often include test descriptions that will prove its completeness when “Done”. — The Scrum Guide.
So as not to confuse anyone… these are NOT the definitions of “Done”. I’ve come across many examples where teams would have specific DoDs unique to a PBI. Definitions of “Done” apply to the overall Increment (and thus to all PBIs being worked on for that increment). Test descriptions can however be unique to PBIs. Many teams will refer to test descriptions as Acceptance Criteria or Acceptance Tests. These describe very specific conditions / needs to be met. In Software Development, these tests can be created before the realization of the PBI; a practise called Acceptance Test Driven Development.
“ATDD (Acceptance Test-Driven Development):
test-first software development practice in which acceptance criteria for new functionality are created as automated tests. The failing tests are constructed to pass as development proceeds and acceptance criteria are met.” — Scrum Developer Glossary (Scrum.org).
These could too be refined with definitions of expected behavior:
“BDD (Behavior-Driven Development):
agile software development practice adding to TDD the description of the desired functional behavior of the new functionality.” — Scrum Developer Glossary (Scrum.org).
Refinement
Now, woah woah woah there! I am getting ahead of myself. Organisations may be tempted to spend a massive amount of time to creating a near complete Product Backlog before development begins.
No! Remember that the Product Backlog is a living document. It emerges as the product itself emerges. Do not fall into the Waterfall habit of specifying all requirements into a massive backlog before either design or development starts. And do not as a Product Owner, single-handedly refine the Product Backlog items with details on what is needed to be done to meeting a requirement, or how this is to be done…
Many misinterpret the following responsibility of the Product Owner:
“The Product Owner is the sole person responsible for managing the Product Backlog.” — The Scrum Guide
The Development Team is structured and empowered to determine themselves how to best go about meeting a need. That said, they do this together with the Product Owner during refinement.
“This is an ongoing process in which the Product Owner and the Development Team collaborate on the details of Product Backlog items.” — The Scrum Guide
One might argue that, as it is the responsibility/accountability of the Product Owner to optimize the value of the work the Development Team does, they get to tell them what to do and how to do it. However, Scrum intends this to be interpreted differently. A Product Owner best does so by utilizing the creative and intellectual abilities of the cross-functional Development Team. The Product Owner will help the team understand purpose, first and foremost.
When the purpose of a PBI is clear, it will emerge through refinement as more and more details are added.
“Refinement usually consumes no more than 10% of the capacity of the Development Team. However, Product Backlog items can be updated at any time by the Product Owner or at the Product Owner’s discretion.” — The Scrum Guide
Note the explicit mention of ‘usually’. It can take a bit more or less, if it serves to keep the Product Backlog in a transparent state.
Definition of ‘Ready’?
Definitions of “Ready” (DoR), counter to what is often assumed, are not exactly core to the Scrum Framework like the definitions of “Done” are. The Guide merely states:
“Product Backlog items that can be “Done” by the Development Team within one Sprint are deemed “Ready” for selection in a Sprint Planning.” — The Scrum Guide
Often the Product Owner and Development Team will require a shared understanding of when a PBI is “Ready” to be selected by the Development Team into its forecast. An example of commonly used DoRs are the INVEST criteria. The guide only has one definition or requirement, and that is that the Development Team should deem it feasible to be “Done” in one Sprint. When DoRs are applied they generally list the work that needs to be done to each PBI before the Product Owner and Development Team can sign off on it.
Definitions of “Ready” are generally a good practice, but they aren’t prescribed. DoR’s may create the illusion that everything is known and clear upfront. At times they could get in the way of a teams agility and productivity. Remember that a PBI doesn’t need to be a 100% clear and that during the Sprint Planning and Sprint itself, the Development Team and Product Owner should continue to collaborate on the work to be done on them. The Development Team will discover new complexities, better approaches, unexpected/unpredictable behaviors, and potential improvements as work progresses. The Product Owner will discover potential improvements and new requirements too. So when DoRs are in play, avoid getting bogged down in continuous negotiation over them as that could be distracting. Teams may be tempted to rigidly ignore all these valuable insights as ‘they weren’t originally specified in the PBI’. DoRs could end up being a cause for delay and incur cost of delay.
Often collaboration is a much better alternative to all this documentation in PBIs. Rather than relying on poorly written/read/executed specs, the PO and Development team will learn to rely on their direct collaboration. At times it makes much more sense to document/specify only what work has been done, rather than the work to be done.
Rather than relying on user stories, rely on continuous storytelling.
So when is it “Done”?
When work on a PBI in a Sprint is “Done”, this isn’t to say the entire PBI is “Final”. Embrace the incremental nature of development. It might take a few iterations before a PBI’s purpose is measurably achieved to satisfaction. When in use, the work that has been done on a PBI is subject to continuous inspection and adaptation. As more is learned, work continues to be performed on it in incrementally.
Ordering
Scrum leaves the ordering to the discretion of the Product Owner, though it does hint that items at the top tend to be more detailed and of more value. All sorts of practices could be applied.
Note that we are using ordering alternative to prioritisation. An interesting update made to the guide, mainly revolving around the idea that priority doesn’t necessarily equals the order for the items to be worked on. Generally I find a top-down ordered list to work best, but that is a personal preference of mine. This instantly makes clear what to focus on, and it doesn’t leave room for ambiguous interpretation of prio like MoSCoW does. With MoSCoW, it is generally my experience that Would Haves generally equals to Never Haves and everyone knows it. There will be tons of Must Haves. That doesn’t create the transparency a Development Team requires for it to be able to focus. When I ask which of those Must Haves the team should focus on, the answer generally is ‘all of them’. Although flagging items as Must Haves is a technique that could be used to keep impatient stakeholders happy at bay, as they’ll appear to have priority. The truth is that this way none of them effectively are.
Scrum.org gives us this visual representation of a possible Product Backlog:
A Product Backlog could/may include projections too, as long as these are based on past performance of the team and the rate of progress made to date, and only as the Backlog currently stands. These will change as the backlog changes over time. The problem is however, that once expectations are set, stakeholders will expect them to be met. This will lead to toxic practices that will make the Scrum Team more rigid, as they will be tempted to satisfy those expectations over adapting to new valuable discoveries.
A Product Owner generally makes these projections transparent to stakeholders during the Sprint Review:
“The Product Owner discusses the Product Backlog as it stands. He or she projects likely target and delivery dates based on progress to date (if needed);” — The Scrum Guide
Scrum.org gives us this visual example:
Product Owners too could take hierarchic approach to ordering ordering the Product Backlog such as Agile Product Portfolio Management (PPM).
The order of the Product Backlog too could reveal possible dependencies between PBIs. They might be grouped together, colorized or linked up and what not. I’m sharing these only as examples and wish to, once more, emphasize that it is solely up to the Product Owner and that his or her choices in the matter should always be respected. The Scrum Master will help the Product Owner by sharing possible “techniques for effective Product Backlog management” and “helping the Scrum Team understand the need for clear and concise Product Backlog items”. Improving Product Backlog ordering could be a welcome topic for any Retrospective too!
I care to share one more possible technique called ‘No’. It shouldn’t be about what should go into the Product Backlog, but what shouldn’t (or what should be removed). Remember though that, when returning a “no”, do care to explain what would be needed to turn it into a ‘yes’ in managing the relationship with stakeholders.
“Simplicity — the art of maximizing the amount of work not done — is essential.” — the Agile Manifesto
Estimates & Value
Oh boy. I am excited to cover these two topics, yet also dreading the exercise. There is so much to cover even though the Scrum Guide has very little to say about it. [I’ve covered this in the next episode.]
Inspections
Product Backlog items could/should contain reviews and metrics of the use of the actual product. It shouldn’t just be filled with assumed needs of guessed value. Many teams are in a constant deploy & forget mode of production. They are always producing new stuff. Even when measurements and inspections are in play, they hardly ever affect the play. Hardly ever do development organisations validate if the expected value has converted to actual value, let alone adapt. Inspection is fundamental to Scrum and its effect should reflect in the way the Product Backlog is organised.
The case against the Backlog
Closing this episode, I want to cover the potential dark side of the Product Backlog raised by Ron Jeffries in: Dark Scrum: The Case Against the Backlog.
He illustrates, as I too have experienced, the backlog becoming an excuse to not having to collaborate directly. He, like myself, dreads the backlog turned requirements document. Ron points out that a Product Backlog could end up counterproductive in situations where Scrum isn’t properly practiced (which, let’s face it, is the vast shocking majority).
An error, Ron calls out, to which I agree to some extend, is that the Scrum Guide calls for the Backlog to contain “all” requirements, so this could even include all the wasteful mumbo jumbo.
Many many many practitioners will often default to traditional behavior of treating the Backlog as a Requirement Document. I often find the Product Backlog stowed away in some barely accessible, over configured Jira or spreadsheet environment, with all sorts of bureaucratic attributes and input requirements, containing poorly written specifications and biased assumptions, linking to numerous external sources.
Dark practices continue like:
- Only doing that which is defined… sticking to the letter.
- Exchange of information only through an artifact.
- Having others than the Development Team define the work to be done by them.
- Creating (or should I say imagining) unreliable projections setting false expectations.
- Establishing process controlling configurations by some overenthusiastic process engineer/admin/manager/product owner/scrum master/agile coach.
These backlogs only ever grow and grow and continue to be the center of toxic political power play. No amount of refinement in the world could possibly keep it all in a transparent state.
Having stated the above, I do still believe the Product Backlog to be extremely valuable. It should be in play and teams should learn to master it. I am talking about ‘Ri’ level insights here.
I personally believe that seasoned Scrum Teams in mature Scrum environments, with all the values and principles embraced… could work from a very lightweight version of it. This could contain only the most imminent valuable items, made visual on a board in the actual workspace. This could contain the only documentation required at the last responsible moment.
There is loads more to cover on this subject. By no means do I think this episode covers the Product Backlog to a satisfactory level. Lots of amazing practices, anti-patterns, and examples could be referred to. I’ll simply refer you to one of my favorite references: Service Design Tools. This site contains lots of useful workshops Scrum Teams can engage in, to help them better understand the product, market, customers, users, and each other.
The Road to PSM III is being reformatted to The Road to Mastery!
Part I: Down the Rabbit Hole is now available.
Next Episode:
