Lessons learnt with a zero bugs policy
Some of our teams have been trying out a zero bugs policy, following a simple remit:
If a bug is reported that you feel you should fix, then you fix it right away. If not, you close it.
We wanted to share what we’ve learnt.
Did it help with issue management?
For us, yes.
Our teams have spent less time managing bugs, and more time improving the products they deliver.
Do all teams follow the same process?
Not quite.
One team hasn’t received many issue reports. They can handle each report on a case-by-case basis, without a defined process.
Another team had far more issues reported. This has led to a more detailed triage process, to ensure we treat all issues fairly.
How do we triage?
We made a checklist to spot issues that should clearly be prioritised and fixed:
- Does it block use of the product?
- Can it damage the user’s environment?
- Is it a regression?
That doesn’t mean all other issues don’t get fixed. We always consider the problem a user is facing and provide the necessary support. We also offer other solutions, like improved documentation.
How do customers react to it?
Reaction has been roundly positive, and people are happy to close support cases with us sooner.
Some are disappointed to not get a fix, but we work hard to ensure that everyone gets the support they need.
How did we get on top of the bugs we already knew of?
Once we could triage incoming issues, we applied the same process to issues that had been reported in the past. We triaged our backlog batch-by-batch, quickly reducing it to only a handful of issues.
This highlighted some bugs that really should have been fixed, which we addressed. Many issues were no longer relevant for our users, so were quickly closed.
Have we rejected any issues that came back to bite us?
Not yet. We take the triage and analysis of incoming issues seriously, and still release fixes for many reported issues. We continue to listen to feedback form users, and hope to avoid misdiagnosing issues.
What challenges did we face?
The biggest challenge is getting that triage process correct.
Having a checklist is useful for quick classification but following it blindly would lead to poor decisions. It’s crucial to do enough due diligence to understand issues and make the right call.
If we decide not to fix and issue that’s been reported, but someone couldn’t explain that decision to a user in a way they’re happy with, that’s a likely sign we haven’t considered the case properly.
Will we continue with this approach?
Yes.
Both teams who have adopted a zero bugs policy approach are keen to stick with it. We’re not forcing all our teams to adopt this approach but will happily support them if they’d like to.
