Paying Down Technical Debt Using Bug WIP at Pet360

The purpose of this article is to illustrate how Pet360’s eCommerce team applied the Kanban concept of “Work In Progress (WIP) limits” to calculate the ideal number of bugs that the team should always be working on. Within Kanban, the idea of WIP limits is to constrain the number of tasks that can be worked by a team. Ideally, the WIP limit represents not just the maximum, but the minimum as well.

The team should always be striving to be exactly at WIP to maintain efficiency. We wanted to apply that idea to the total number of bugs that should be on our Kanban board at any given time. Specifically, we are talking about bugs, otherwise known as escapes, that are defects found in Production.

It should be noted that bugs found while working on a project are immediately addressed and included in the existing scope of work, and were not included in our calculation of WIP because they did not make it to Production.

We wanted to use our existing Agile methodology to help inform the number of these escapes that the eCommerce team should be expected to work. It was decided that this is the average number of bug stories the team historically worked each week. This way the precedent was already set by eCommerce engineers and Product Management when they agreed upon what went into the short actionable list of work to be pulled — which we call the Ready Queue.

Over the past three months our work and bug tracking tool, JIRA, has shown that the eCommerce team has worked 37 bug stories.

  • 37 bug stories divided by three months equals 12.3 bug stories per month.
  • 12.3 bug stories a month divided by 4.4 weeks in a month equals 2.8 bug stories per week.
  • 2.8 rounds up to three bug stories per week.

In other words, the eCommerce team always wanted to make sure that we had three bug stories, either in the Ready Queue or in flight, at all times. This means that team leads and Product Management need to be mindful of this number when back-filling our Ready Queue.

The eCommerce team currently maintains a backlog of outstanding bugs that hovers around 40 stories (38 at the time I wrote this article to be exact). If you strip out the bug stories that are older than six months, that number currently goes down to thirteen. The idea is that all unaddressed defects in Production older than six months have proven themselves to not have had a high enough impact to be addressed by the organization and should be ignored.

We are also using a JQL filter in the JIRA Reports feature to track the rolling monthly average of bug stories created by the eCommerce team by week. Our desire is that this metric is going to help prove our “quality over speed” stance. If we keep working down bugs, and generate few, then the ideal state of Bug WIP is attainable. A positive number means that the team is generating more bugs that they are closing. A negative number means that the team is closing more stories than they are generating. Negative numbers are ideal.

I am hoping that by sharing Pet360’s experiences I can pass on mechanics and tooling for other organizations to utilize. We would love to hear from other organizations and what worked for you.