Ticket Weighting

One of the common issues I’ve witnessed during the many years of being a part of software development is the way we handle and classify not only bugs but also all tickets. We are now living and breathing agile and I think we need to adopt a much smoother, transparent and efficient way of classifying tickets.

Ingredients

• Jira or your favourite ticketing system

So, where are we now?

At the moment I see a somewhat of a common trend when speaking with others within the software development field (mostly the video game sector) as well as my own personal experience of 11+ years in the Games QA sector is that we classify tickets on a two-factor scale:

1. How severe the bug is. If its a 100% reproducible software crash it’s most likely class A
2. The priority of the ticket or issue. For example with the above-mentioned crash, this could be prioritised as a class A or Blocker or 1 (depending on which scale you have adopted)

and that is generally it. Usually, the PM, PO or Lead Dev would utilise these two scales to define when, how and who will resolve these tickets.

So, what is the problem?

The main problem is that the above is something which was brought over from the boxed software world, and when you’re working in SaaS industry it doesn’t translate to how we work and how we listen to our customers and also how we observe the impact on the revenue of the product.

When you’re planning the next release and you look at the backlog, how can you see the impact on the following factors using the above two-factor scale?

• Technical Scope
• Revenue Impact
• Customer Impact / User Pain

The answer is, that you would either have to get the Developers to write something in the ticket to explain the technical scope, you would then have to get the Production team (PO, PM) to change the priority based on the business impact in terms of possible revenue decline and lastly, you would need to either ask the customers to rate the ticket or have the Customer Service team to write a comment or to also adjust the priority.

The above method is usually attained in some kind of bug triage meeting and most of the time takes up a lot of time from the respective representatives from the departments and can if not moderated strongly derail in emotional conversation on the tickets that end up not moving the topic forward.

So, how do you plan to fix this?

The answer even though is a simple one, requires you to step back a little and think about the bigger picture as well as think about the product maturing and how we can make sure we have a stable and efficient approach.

What I propose is ticket weighting below is how I would tackle this issue, again you need to keep an open mind and don’t let the excuse of ‘there is no time’ block you from trying new things.

The idea is this:

The ticket has 4 numeric rating fields:

User Pain – Impact on the customers experience

• This can be further refined by saying 1 would mean less than 5% user base impact up to 5 which would be 90%+ impact

Technical Complexity – How hard, complicated and time-consuming is it to work on the ticket

• This can be further refined by defining something like classification 1 would be it would take one developer less than half a day to fix and a 5 would require a team of developers multiple days to fix for example
• If you’re already using a measurement for complexity (like Story Points or Time Estimations) you could use these values instead for the technical pain, but the formula would have to be adjusted for the overall ticket rating

Business Impact – Does this ticket have any impact on revenue

QA Pain (bug) – What is the type of bug (crash to cosmetic)

• There should be already a definition in place to classify bug tickets from the QA perspective. But you could do something like:
• 1 = Minor Text and Minor Graphic Issues
• 2 = Localisation outside of text area
• 3 = Missing translation
• 4 = Low % Crashes, More Annoying Graphical Problems and Performance Issues
• 5 = High % Gameplay Blocker, Legal Issues or Data Loss
• The higher the ticket value the bigger the impact it is to the product and customers

So, how does this look?

It’s easier if there are a couple of examples to demonstrate this approach in reality.

Example 01:

Let’s say we have a software crash which is not 100% and is not on the main user path. This is how this ticket could look with the scoring:

User Pain – 2 as this specific issue is off the main path for the user and our Customer Support team informs us that this is not a high impact area

Technical Complexity – 3 as this would take a single developer 1 – 2 days to fix

Business Impact – 1 as this has no revenue impact on any events, monetization features and is off the main path

QA Pain (bug) – 4 as its not a 100% crash but a crash is rather severe

In the end this specific ticket would score: 24 / 625

–

Example 02:

Then, let’s say we have an issue with premium currency not being awarded after the Christmas quests have been completed, this is what the scoring could look like:

User Pain – 5 as most of the users would jump on this event and not receive the communicated reward causing frustration and pain

Technical Complexity – 2 as this would take a single developer 1 – 2 hours to fix

Business Impact – 5 as this was an event that the user needed to utilise premium items which cost money and affects the loyal customers

QA Pain (bug) – 4 this is a progression blocker

In the end this specific ticket would score: 200 / 625

–

Example 03:

Let’s say that there is a minor feature thats multiple clicks away for the user and this feature is something popular. The users would like it on the main UI so its more accessible:

User Pain – 5

Technical Complexity – 1

Business Impact – 2

QA Pain (bug) – 3

In the end this specific ticket would score: 30 / 625 – But looking at the various metrics you can see it has a very low technical complexity for fixing the issues but the user pain is high. This kind of issue would be a quick win and show the user base that we’re looking after their interests.

–

If an issue ever gets to 625 / 625 then you know you have a problem. The higher the score the more it needs to be tackled (and quickly)

So, we have a score now what?

There are a number of different things we can utilise with tickets all having various values. We could for example purely look at all the tickets that have a high user pain or company pain. We could then show all the issues with low technical pain, this then would show us quick wins for our product. This approach has many different applications, but the one I want to focus on is tracking the overall product quality.

As a company, we may define that we want all our products to have an open ticket score of let’s say 8,000 points maximum before being able to release. This means we are unable to release an update to the product if our total open bug score is greater than 8,000 or we may define that we have several products at different maturity level with different teams and focuses and apply each team with their own score.

What you can also do is track this over time and trend it, below is an example graph based on the above information:

As you can see from the graph the indication is that the product’s overall quality is improving. With this data we could drill even further and see which areas are causing the most pain based on QA Pain (definition of the type of bug it is) and correlate it also with technical pain and the work that went into this version. The idea is that we have a general indicator which we can drill down into and identify problem areas and to also visualise how we are doing as a product.

So, what are some of the other benefits?

There are so many positive side effects with adopting such approach. Below are some examples:

From a Project Management perspective, I believe you should also apply this to all task tickets (stories etc) because this could identify quick wins in the backlog and enable you to actually see the product from a very good point of view. You could use the approach that this sprint, week, month, timebox etc to only focus on technical pain or QA pain and create a quick JIRA filter to display this data and let the teams self-organise.

You can also create a JIRA dashboard which shows the overall score of Bugs and the Backlog and then break down each category. I believe this would give you a really good overview of the product. You could then also define limits per category for example, the user pain score cannot exceed X amount.

From a Release Management perspective, you would get a good overview of the release as well as having a very good base for informed decision making which is transparent enough for management to also get a clean overview.

From a Productivity perspective, you could even gamify the effort, as the tickets all have scores and the higher the scores the more important the issues are, you can see how many points people have helped to erase from the products overall score. For example Tim the developer worked on a number of tickets this release that have an Impact Score of 1014, you could award prizes for reaching a set score per month.

If you’re really living and breathing an agile approach to team management and want to push the boundary even further you could have your project manager fade more into the background. Everyone in the team from developers to artists can always take from the top of the pile, the tickets with the highest scores. You can have a filter that splits work up by internal department, such as Game Design, Development and Art for example. Combine this with a release train and you will have a very automated development process which is fast, agile and empowers everyone in the team.

Ticket Weighting, the conclusion

In this ever changing world it’s important we do not cling onto the past and we look at evolving not only processes and management but we also look to see how we can make life so much simpler. I believe that any product that takes the above approach will have transparency on every ticket, they will have a better understanding of their product from a quality point of view and can identify quick wins so much easier. But don’t be fooled, this is not the golden bullet you were looking for, this approach isn’t perfect and required the team and management to not only buy into the approach but to also live it.