To estimate or not to estimate bugs?

Should defects be estimated or not? I stumble across this topic again and again. I have a clear opinion. But my team also has, and their arguments are valid.

I’m preparing an estimation workshop for a new team. The aim of the workshop is to explain the concept of Story Points and relative estimation. In this context I also prepare myself for the question whether defects should also be provided with Story Points or not. There are very different views on this topic.

I say: Bugs should not be provided with Story Points in order not to “screw up” the velocity. This is true if you have defined velocity as a measure of how much functionality a team is able to deliver in a Sprint. Furthermore, it’s called Story Points because you estimate User Story (value), not bugs.

My team says: There is technical debt that needs to be quantified in order to start a conversation. The technical debt doesn’t necessarily have to be legacy bugs, but also outdated components from third-party vendors that are no longer maintained, and thus have reached the end of their life cycle.

PRO estimating bugs

  • the true abilities (Story Points per iteration) of the team are made visible
  • technical debt gets a quantitative value
  • estimations can be updated to represent the interest rate

CONTRA estimating bugs

  • the Velocity no longer represents the progress of product development
  • bugs are often so small that it is not worth estimating them, or
  • bugs are so complex and risky that it makes no sense to estimate them in Story Points (I prefer to timebox them)

Individuals and interactions over processes and tools

Nick Brown mentioned something at LKCE18 that became one of my guiding principles: “Use metrics to start meaningful conversations rather than using them as targets”. And I think he’s right about that. So if a team decides to estimate bugs in Story Points (“Bug Points”), and include them in the Velocity, then that’s OK. However, they should keep the ability to detect a performance decrease and address this in the retrospective meeting (or any other time) if needed.

Focus on business value

This is very easy if you have provided the product backlog items not only with estimates in story points, but also in business value. The Return on Investment (ROI) is calculated simply by dividing the Business Value by the Story Points for each “done” story and thus determining the value “Business Value per Story Point”.

A prerequisite for this is that business value estimation is not done by the development team, but the real customers (or their close representatives). I suggest to use the current “Dieselgate” as an example: Are the buyers of modern VW diesel vehicles willing to pay money for hardware retrofits, so that the promised emission rates can be met?

Under the assumption that a customer is not willing to pay money for a bugfix, the ROI for this bugfix is significantly lower than for a new feature. This circumstance is also reflected in the ROI of each Sprint, as shown in this fictitious example:

Velocity (incl. Bug Points) vs. ROI

Maybe there is a Product Owner out there who is able to increase ROI while the team increases the amount of “Bug Points” per iteration. In this case, counting Bug Points into the Velocity again would be a missed opportunity to identify early warning signs for a slow down due to technical debt.

Do I have a solution? Yes: Talk to your team, find a common understanding, and make it transparent to the ouside so everyone knows how to read the numbers.

I recommend to read these articles as well:

Note: I’m not a native English speaker. In case you find something wrong, misleading or unclear, let me know. If you have a suggestion to change the wording — yay, let me know! I see blog posts as a living object which is subject to change.