How we went from a 350+ Bug backlog to an almost Bug free Product

Introduction:

Have you ever been in a team where there are 350+ bugs in the product’s backlog, and to make things worse, new feature developments come with even more bugs?

Or in teams where management and other stakeholders are worried about Product Quality after every sprint or project release?

You’ve probably come across PO’s, EM’s, LD’s, QA’s, and developers who don’t have the solutions or motivation to improve the situation.

On any given day, most of us deal with circumstances like the ones mentioned before. If you’re lucky, you’ve only heard stories about “an unfortunate team” in some faraway company struggling with an unfortunate predicament like that. Well, a few months ago, we were that “unfortunate team” you’ve heard about in those stories. Our product backlog really did have over 350+ defects, and the list was only growing.

Curious to know how we got the situation under control? we would like to share our success story with you. You’ll learn how we managed to stabilize product quality and learn some valuable best practices we discovered during this incredible (and arduous!) journey.

What is a Bug / Defect and how does it impact quality?

Before diving into the topic, let’s understand what is a bug or a defect and how it impacts the quality of the product.

A bug or defect is an unwanted behaviour from the software system which does not adhere to the business’ needs. It could be anything that causes a mismatch between the “Expected” and “Actual” results during any user interaction.

System quality is nothing but the degree to which the delivered system meets the requirements specified by either the Business or the end-user’s needs and expectations.

In simple terms, bugs are the units which we use to measure the Quality of the software system.

Imagine that our product , let’s call it Product ABC, has over 200 bugs in its product backlog; including different issues like Functional, Non-functional, UI problems, UX issues, etc. In this case, Product ABC’s Quality would be rated in “Red” Zone depending on the priority and severity of the bugs.

On the other hand, there is another product, Product XYZ, with around 50 bugs in the backlog but no critical issues on the site. In this case the product quality would be rated in the “Green” zone — considering that all other dependencies are equal on both projects.

There are various algorithms to precisely set standards and “danger” zones based on various parameters and methodologies. But these are two simple examples which explain the relation between bugs and quality.

What are the key pillars of Quality?

Now, let’s classify the key pillars impacting the site’s Product Quality. Every development team needs three key pillars to monitor and maintain a high level of Quality. We used these three pillars to find our way out of the bad spot we found ourselves a few months ago!

1. Maintain a Healthy Product Bug Backlog.
2. Uphold Quality Standards for every new feature development.
3. Assess the Health of a component on a regular basis.

1st Pillar: Maintain a healthy Product Bug Backlog

Maintaining a healthy product backlog is the first stone you need to set if you want to build a successful product. During the product discovery phase, it’s quite normal that quality could be compromised over new feature releases that will increase the market base. However, if members, QA’s, and stakeholders raise numerous bug reports on a daily basis, the bug backlog piles up and eventually key stakeholders like PO’s, EM’s, LD’s, and QA’s need to take dire steps to improve the quality. Let’s take a look at how to organize the roadmap so this doesn’t happen:

Start by splitting the bugs by product vision and components like Priority, Business criticality, Product…

Obvious Red Bugs

Start creating the respective EPIC’s based on the roadmap. Its advisable to have a dedicated EPIC for bugs on Quarterly basis (Eg: Product — Q2 — Red Bugs). Plan and move all the obvious Red bugs under this EPIC, which are most crucial to improving the Product Quality. Based on stakeholder feedback, a dedicated team or colleagues could be assigned to fix these within relevant timelines.

Older Red Bugs

QA colleagues are great assets and they play a crucial role supporting the creation of these bug tickets. It’s always advisable to mention the Reproducible rate in the tickets if the bug is not 100% reproducible.

As a Product owner, start filtering the bugs which are not 100% reproducible and based on the number of bugs and their created date, start allocating the bugs to different EPIC’s or labels so that the QA team can try to reproduce, retest, or close each bug before organizing the Defect Triage or Sprint planning.

Defect Triage or Sprint Planning

Once the list of valid bugs are allocated to different EPICs, the next step is to plan the work of bug fixing. It’s advisable to organize a defect triage or dedicated meeting to discuss the time estimates and brainpower needed for each of the EPICs or tickets. Based on the team’s velocity and estimates, a roadmap with the list of EPICs or tickets could be shared with the stakeholders. As a best practice, It’s always recommended to allocate 20% of the sprint composition to bugs in case of an Agile development lifecycle (if the product is stabilized).

Dashboards

Once the roadmap includes all the upcoming sprints and the allocation of bugs, go ahead and set up a transparent dashboard using JIRA, Easy BI or HP ALM to make the results live and the roadmap visible to all the stakeholders. It’s always recommended to publish the Quality reports on weekly or Bi-weekly basis with the results of the stats at the required level.

Please keep in mind that all the team members including EM, LD, PO, QA, and Developers need to contribute equally during this pipeline and everyone needs to be equally motivated and charged to avoid any team conflicts. Keep in mind that everyone is working to improve the Quality of the product.

2nd Pillar: Uphold Quality Standards during the daily deployment pipeline

It’s always important to motivate and encourage Developers to go that extra mile during the development phase in regards to the Quality of the pull request.

1. Meet with Developers in 1-to-1 sessions, Retros, or regular weekly catch-ups to highlight the importance of unit testing and dev validations before the QA phase.
2. Motivate and encourage Developers with some challenging Algorithms (example one explained below).
3. Setup Dashboards to show results transparently during dedicated meetings or retro meetings.

Encouraging Developers to deliver with Quality:

To appreciate and motivate developers who take that one extra mile in maintaining the Quality developments, initiate some programs inside the team to have a healthy competitive team spirit. Here is an example below..

• The intention of this program is just to reward and motivate colleagues.
• This needs to be handled confidentially by EM, LD, PO before all team members are on the same page, to avoid criticism or conflicts inside the team.
• Performance differences should be sorted in 1-to-1 meetings before the roll out of the program.
• A newsletter or formal appreciation during a dedicated meeting makes top performers feel appreciated.

3rd Pillar: Assessing the health of a component on regular basis

Assessing the health of each component on a regular basis plays a key role in assessing the overall health of the product. After all, a product is nothing but a conjunction of components working together. So, as a Product team, it’s essential to always monitor the health of each component in the Product.

It’s always advisable to do a brainstorming session with the team to understand the potential improvement areas in a Product. If there is a specific component which is too buggy and causing a lot of regression at Product level with each deployment, it’s advisable to refactor this component in a strategic way. This definitely needs support from the Business team so that they can prioritize and allocate the respective sprint, since this is a technical initiative that benefits the product by improving the overall product quality. The below flowchart gives a high level view on how to perform this activity along with all the stakeholders.

Conclusion:

These are the best practices and key pillars we discovered and implemented to stabilize the product quality.

Although its an obvious fact that software products don’t exist without bugs, it’s quite important to make sure that these bugs are monitored on a regular basis with relevant strategies before they impact the reputation of the site and the product team.

Terms to know:

• Red Bugs: As per the standards, Bugs with High, Very High Priority are classified as Red Bugs, which are apart from Blockers.
• Pull Request: An event where a Developer contributes to the system change with a specific version control which has the change request of the code.

………

Written by:

Hari Janapareddy: Product Quality Manager

Ryan Balazadeh: Development Product Owner

Thank you Oguz for the suggestions regarding this article. Your tips and pointers took this article from rough draft to a technical paper that we are proud to publish.

Thank you Fernando Garcia Caraballo for your writing advice and editing skills.