SummerAcademy on Debugging — 2.1.1 Issue Tracking Systems

Find the companion video here.

Originally published here https://www.deepcode.ai/blog/summeracademy-on-debugging-2.1.1-issue-tracking-systems

The first kind of tool we want to talk in the tool chapter of the Summer Academy is issue tracking systems. We see the three major advantages of a tracking system: First, collecting the issues in a structured way. Secondly, forcing and tracking a process. And lastly, providing the opportunity for release planning, retrospective, and BI.

2.1.1.1 Collecting Issues

Simply put: All issues should be handled through an issue tracking process using an issue tracking system. Reject to address any issue not captured within the issue tracking! For persons who cannot (or too senior to be asked :-) ) file a report, offer to do it for them. Make sure all those who should be able to report an issue are well aware of the process and know the rules of the game.

Getting hold of an issue tracking system is not that hard. GitHub and GitLab provide basic functionality. Lots of teams use JIRA and there are open-source alternatives such as Bugzilla, Launchpad, OTRS, Redmine, or Trac.

A good issue report is actually harder done than said. Here are some best practices for good issue reports:

• The best issue report is a fully or at least partly automatic one that collects all necessary data points, informs the user about this, and is filed. Automize as much as possible.
• An issue report needs to contain a precise description of how to reproduce the issue. Following the short, self-contained, correct (compilable and runnable) example (SSCCE) (see SSCCE)
• Provide a template and good guidance is key and needs to be followed religiously. Here the minimal fields:

Regarding the severity heuristics:

We would assign a weight following these lines (from high to low):

• Safety means a person’s health or life is endangered.
• Security means the confidentiality or integrity of the system is lost. Possible consequences are dramatic.
• Data Loss undermines customer trust which is hard to regain
• Availability reflects downtime that can normally directly be converted into money lost but also impedes lost trust and lost customers
• Crash or Freeze is again a source for data loss, customer annoyance or even losing users
• Ops Issues reflect problems that are suppressed by ops activities such as memory leaks or exhaustive use of resources. Ops can temporarily suppress the issues bubbling up to user awareness but it adds complexity and possible failure scenarios.
• Code Hygiene addresses
• Legacy Support means supporting outdated environments either from a hardware or software perspective (Windows 95 and Internet Explorer anyone?)
• Backward compatibility captures how well you are doing with older versions of your product
• Cosmetic Issues like a truncated text in a help file.

Qualifiers:

• Affects # of users
• Affects VIPs
• Documented Workaround exists
• Rarely used features

You might want to ask for possible security breaches (always highest severity!!!) or customer data loss, reduced availability, reduced performance, etc. You can multiply by a weight factor given by the number of impacted customers or the importance of the customer, the age of the report, etc. Taking the age into account also resolves the issue of reports falling through and effectively never being looked at. Assign numbers on a spectrum that makes it unlikely that they will overlap so you can easily sort by this number and start to work from the top during the triage.

Affected Shareholders are interesting to understand the extent of the issue as well as whom to keep informed about the process, ask for further details, or to test possible fixes.

There might be the need for an Expedited Bug Process to address the immediate danger to top goals like safety or security

2.1.1.2 Force into Process / Track Process

• One aspect of the Issue Tracking is to force all participants into a clear process. Believe us: While none of us likes to work in a process dictatorship, without process we suffer even more.
• Triaging is necessary as seldomly we have enough people and time to address every bug. See above how to support the triaging. Triaging also means to make a decision, argument over it, and then stick to it.
• Timers are extremely important as bug reports tend to fall through cracks or get delayed indefinitely. Rather than delaying constantly, make the decision not to do it. That gives at least a clear expectation.
• Who reviews when what are the options. A clear process helps here again. Define the Who, the When, the Possible Outcomes.
• Information flow to externals (e.g., reporter, sales, marketing). A bitter message is better than no message. Automate information flow so that possible stakeholders are informed and can authoritatively communicate.

2.1.1.3 Retrospective and BI

If you do your Issue Tracking well, the data can be a treasure trove. But be aware: Garbage in, garbage out. Make sure the data quality is good. Always prefer automatic data over manual entry, because it is an additional burden and most of the time, the quality degrades over time.

• Get an idea about team performance by using statistics from your Issue Tracking. Having a good idea of how much stuff you can load into your sprints, is the sign of a mature team.
• Quality measures, especially on legacy software, is quite interesting. Your Issue Tracking can provide a good idea of what problems still exist and who is suffering, how long it takes to react on issues, etc.
• Identify problem areas like libraries, parts of the app, not understood features by combining data from the Issue Tracking and the Version Management (see 2.1.15).
• Review your issue tracking process from time to time and apply changes. While it is necessary to keep strict with it, it does not mean it needs to change. Ask every stakeholder. If you change, communicate to everyone and automate as much as possible.

Key Take-Aways

Issue tracking is essential. Make sure to have a rigid format and process at hand. Use heuristics to reduce subjectively (and sometimes hard) decisions.

Resources

Comparison of Issue Tracking Systems Best practice discussion on StackExchange