Good practices for backlog ordering

Every day, working as the Product Owner in a software company, I receive several inputs in terms of suggestions or requests for the product that we are building, on top of what is already on our radar.

The ultimate goal of the PO is to produce and maintain a single list of things properly ordered, namely the backlog, that is the basis for the periodical sprint planning with the development teams. If the PO does this job well, having the teams work on the items in the highest positions of the backlog allows the company maximise the value added to the product, for the invested effort.

In this article I’d like to share a few good practices for backlog priority management, successfully applied at Funambol over several years, that are relevant in broader contexts as general rules.

“Extended” benefit-to-cost ratio

The benefit-to-cost ratio is surely one of the main and first criteria to consider during the assessment of how far up in the backlog a given user story, for example, can be positioned: user stories with high benefit and low cost normally occupy the top portion of the backlog.

Although a rough evaluation of this ratio is apparently quick and easy, too often an oversimplification leads to unpleasant surprises and subsequently waste, delays, extra efforts.

It is therefore recommended that the scope is a bit widened, looking for any side effects that such a change could introduce, while still considering the benefits.

As an example, let’s assume that we are analysing a user story about an improvement to a flow in section A of our app: considering only this section, it may appear that we could make the user happier for very limited cost. However, a deeper analysis might reveal that the same flow can also be triggered from sections B and C, where the improvement in the user story cannot be applied for one or another reason.
What could we do? Reject the user story? Lower its benefit rating? Should we break consistency between A, B and C? The latter may even be justifiable, considering the gain of the improvement that we would introduce, but it is surely important to be aware of it. Many times, the PO is not alone in making decisions like this: a broader group should be involved, if possible including the stakeholder that might have proposed the change, as well as any other impacted stakeholders.
Obviously, if A happens to be the core of the app, while B and C are peripheral sections, it’s much more likely that the user story would rank higher in the backlog.

Looking at costs, one of the most common mistakes is to only consider the development effort: instead, the PO should also factor in what is spent to analyse and design the feature in advance of development as well as the cost to maintain and evolve the feature post-development.
It is not a rare occurrence that a deeper upfront analysis results in lower future maintenance costs for the feature, delivering higher economic benefits for the business as a whole.

Let’s take another example around consistency, involving a user story for building a small feature.
Regardless of how simple the feature, there are still some common aspects and patterns across all of the app features that should be considered and followed: how to communicate to the user, error handling, how to behave when there is no internet connection or low battery, boundary conditions, and so on.
Without proper analysis and teamwork, and especially in circumstances with developers not entirely familiar with the product, it is likely that developers will implement such cases according to what they know or like. This can rapidly turn the product into a patchwork of patterns and policies. This situation is not necessarily visible to the end user, but it would either require additional refactoring to get the feature aligned, or extra maintenance effort, not only from the development perspective but also for product management, technical support, etc.

What we learnt in order to prevent these cases as much as possible is to regularly go through the backlog refinement activity. It means to spend a short amount of time to talk about the user story together before working on it, to stimulate and challenge the discussion, to better evaluate what is known, and to raise the unknowns.

To wrap up, the goal is not to arrive at the planning with a very accurate estimation of the work, or to numerically compute the ratio: indeed for the sake of this exercise both the benefit and the cost can be expressed using t-shirt sizes.
The goal instead is to have a good understanding of the overall effort and complexity behind the implementation, and of the value that the item would bring to the product.

Goals ranking first

While the backlog items considered during the sprint planning are about improvements usually small enough to be done in a single sprint, each of them matches with (at least) one high level goal.

These goals can vary, for example, from increasing user activity, to building a new feature as defined in the roadmap; from reducing the costs for running the service, to raising the onboarding rate. Whether or not they are officially set and shared with some techniques like OKR, the company should decide which is the current main goal for the product, and which are secondary.

Once the goals are clear and clearly sorted, this sorting can be somehow translated to the backlog: user stories strictly matching the primary goal normally occupy the top portion of the backlog.

Therefore, when evaluating where to position an item, the PO should understand what is the goal that the item would achieve.
Sometimes this is naturally evident as the user story might have been generated through an impact mapping activity: a top-down exercise that starts from the goal to identify what to do in order to reach it; other times the process is bottom-up: starting from the user story, emulate the 5-whys technique to determine the goal that the user story would lead to.

In a backlog of medium complexity, for a product with some already implemented and deployed features, and multiple stakeholders, it is not as common as it might seem to have all of the items matching the primary goal on top, ordered by benefit-to-cost ratio, followed by all of the items about secondary goal, and so on. The reality is trickier, and this is why there are more criteria to consider.

Focus on closing the open doors

A good backlog ranking also aims to maximise efficiency. The PO should not only pick the right topics to work on, but should also make sure that those topics bring value to the product as soon as possible.

Here the keyword is “focus”, and a metaphor that I like to use is about doors: a team cannot afford to have too many open doors at the same time, open doors should be closed before opening new ones.
You can replace the word “doors” with features, with epics, or more generally with topics: all of them usually consist of several user stories.
There are many reasons behind this recommendation, but perhaps the best known one is to reduce context switching as much as possible in order to achieve higher efficiency.
In addition, as the velocity of a team is limited, the more open doors there are, the lower the velocity per door, the longer the doors take to get closed, and the later the value brought by the doors is added to the product and perceived by the user.
Yet another advantage of focusing on as few open doors as possible is in terms of quality: it is natural that with many topics being worked at the same time, the level of attention that can be dedicated to each is lower, also in terms of analysis, increasing the likelihood of issues discovered later on.
The team should keep the focus as much as possible on the open doors, until the work done can be considered sufficient for closing that door, and then elect a new door on which the team can switch to: user stories related to ongoing topics normally occupy the top portion of the backlog.

Two of the key concepts in agile, the Minimum Viable Product and the Definition of Done, are very related to this good practice: the condition for closing a door is that all user stories required by the PO to reach the MVP for that door, are marked as done.
One of the worst situations, both psychologically and in terms of waste, is when a door is closed, a new one is opened, and the team has to reopen the closed door because of lack of quality: either because some key pieces were left out of the MVP, or what implemented was not actually “done”. It is a much better investment to spend some time to carefully evaluate if the door is actually ready for closure.

There are cases where it may appear as if a door is being consciously reopened, for example for a feature successfully perceived by the user base, where the company decides to invest more: however, in such cases, it is to be considered as a new door and should go through the ranking from scratch.

Finally, as it is not realistic to assume that there is only one open door at a time (unless the development team is very small), it’s also not realistic to be able to postpone all the emerging requests until the ongoing doors get closed: for example, for a high-priority bug in a production environment, or for important business-related activities with strict deadlines, it might be required to make an exception to this rule.

Maintenance versus evolution

With maintenance I refer to the activities to reach a solid and user friendly product, with the current feature set, while evolution is instead the work for making the product richer, adding more features.

Although the latter is generally more attractive, it is a sign of maturity to understand when the sub-performance of the product on some aspects are not due to lack of features, but it’s the current feature set that is not expressing all its potential. Before adding more features, that would require additional maintenance, it is wise to make sure that what already built is performing well: key bugs and user stories related to maintenance normally occupy the top portion of the backlog.

Maintenance includes different types of activities.
Some are more obvious, like behavioural bugs, which are visible to users.
Then there are issues or areas for improvements, that affect the backend, leading to higher costs or potentially serious problems: consider for example data security.
UI/UX improvements are intended to make the product appear and flow nicer and smoother.
Under maintenance I also include technical debts, which are normally raised to PO’s attention directly by development teams.
Finally, the activities to keep pace with the new operating system versions should also be factored into this category.

It is usually harder to tackle maintenance backlog-wise, as it also depends on which stage a product is in: one of the most common approaches is to dedicate some bandwidth in terms of story points, for example, to maintenance, and leave the rest to evolution; or to reserve some slots to address specific areas such as tech debts.
However, in a real-world scenario, these bandwidths might be challenged in several different ways: as per above, some bugs in production or deadline-driven deals cannot be ignored; when the quality metrics raise a warning, the slots or the team’s general approach may need to be temporarily revisited; what emerges from analytics or from user observation can also bring in unforeseen work; and, when it’s time, it’s hard to estimate upfront how much is needed to support an upcoming OS version, but it’s something that takes very high priority nonetheless.

Other factors to consider

There are then other factors that should be taken into account, either strictly coming from the theory of agile software development, or simply from good common sense.

For example, when building a new feature, or even a new product, the recommendation is to follow an iterative and incremental approach: so, the PO and who is working on the feature, should first build a sort of sub-backlog with the ordered user stories for reaching the MVP and then possibly getting extended. This sub-backlog needs to be then blended with the other pending activities, if any.

That said, if this new feature has aspects that present significantly higher risk, in terms of performance, feasibility, or even likeability, it is recommended to frontload them, maybe in a prototype-oriented task, to reduce waste.

Finally, there are things to do that are so small, harmless, and yet deliver value even if their benefit is not game-changing. Such items don’t even need to go through deep analysis, or stay in the backlog for weeks, making it longer and nastier to deal with: in these cases, it is much better to get them done as soon as possible.

Conclusions

The criteria mentioned in this article are just a few coming from several years of real-world experience and lessons learnt, from a pragmatic perspective.

While the ultimate goal is to produce and maintain the backlog, in order to do that the PO must primarily gain sufficient knowledge about the product, the features and then the incoming requests, to be able to perform relative comparisons for a proper sorting of the backlog, and to reduce the risk of bad surprises once the work gets actually started.
The ranking remains somehow subjective, and needs to be reviewed on a daily basis, as new inputs arrive and the conditions change, in pure agile spirit.

The PO should be able and open to explain any choice made, and listen to challenges (e.g. why issue A was ranked high and why B got low priority?): with clarity in mind that priority to A means penalty to B and vice versa, if there are actually good reasons, review the assessment and adjust the backlog.

Despite initial appearances, the PO role is no solo act. Mutual trust and great communication and collaboration with the teams, stakeholders, and the rest of the company, are key for a successful execution of the role.