How to write a good bug report
By DJ Larkin
Originally published May 3, 2013 3:45 pm
Many companies have different names for what they store in their bug tracking system. I’ve heard them called tickets, cases, defects, issues, or tasks. Here at B-Line Medical we call them cases, and that’s how I’ll refer to them in this post. Regardless of their name, it’s important that they’re clear and well written. I’ve covered this in an earlier blog post, but I wanted to go in to more detail on what we consider to be a well-written case.
I equate good cases to a recipe. Anyone should be able to read it, understand it, and work from it. A good case can be returned to months after it was created, and will still be clear and understandable. Cases should never rely on information passed through an email, conversation or IM. All relevant information should be documented in the case, and it should stand on it’s own.
Writing a good case isn’t easy, but it’s important. So, how do we do it?
Language
First and foremost, your case should be written with clear language. Capitalize, punctuate, spell check and use proper grammar. This is a no-brainer. If your case is hard to read, it may cause information to get lost or passed over. You don’t want this. Be succinct, summarize your points and don’t use unnecessarily flowery language. You want to get straight to the case’s relevant information. Don’t lose the point of the post while doing this, though. You want to have the right balance of short and meaningful.
Structure
Your case should be broken down in to sections, and each of those sections should play a different role. If you’ve got places where redundant information needs to be entered, it will lead to problems. Don’t duplicate effort, and don’t create situations that are prone to error. You want a workflow for creating and reading cases that allows the most information to be passed on with the least amount of effort.
Title
The title of your case has a higher visibility than any other portion of the case. It’s going to be seen in searches, listings, and overviews. It’s going to be included in email notifications about cases, and summaries in patch notes. Developers, QA, support people, and anyone else looking at their own queue will see case titles first. Don’t take this lightly. A poorly-written case title can make one problem look less important, or look just like another problem. A good title can be the difference between a case getting the proper attention, and it getting pushed off to a backlog. Essentially, the title is your case “at a glance.”
So in order to make it meaningful, you need to do a few things. The case title should be specific to the problem at hand. It should contain a unique portion of the error, specific to the case, so that it is easily searchable. “Login button not working” or “Error logging in” are bad titles. They’re not descriptive, not unique to the problem at hand, and don’t use details from the actual error. Searching for this specific case would be incredibly difficult If you’ve had a hundred, or a thousand, cases about problems logging in.
Where possible and relevant, add portions of a stack trace to the case title. “Null ref exception at XYZ after clicking the login button” is a much more descriptive title. It gives indication of what the problem is, and can be searched on. It will be descriptive enough for a summary or a case listing.
Some situations call for a naming convention for case titles. This is not appropriate for all cases, but will help out in certain situations. If there’s a new feature that’s been implemented, for example. It helps to have a reference to a feature when there’s going to be a group of cases. This makes assigning them to the right person easier, and makes overviews of case titles group easily. For example, cases created while testing ModuleA could be prefixed with a “ModuleA -> Case Title.”
While you can infer the area of your product that this references by reading the case title, that requires actually reading and processing each title individually. Having a convention will cause you to naturally group these cases together without actually needing to read the titles. This will allow you to process related cases together, rather than jumping around between different areas. You might note most bug trackers have an area dropdown to indicate the part of the product each case is referencing, and my suggestion for case names breaks the rule about redundant information. This is a good point, and why discretion needs to be used when naming cases. Putting an area can denote a subset or a superset of a particular area, giving you more granularity in your case groupings. For example, if you have an area for your permissions screen, but you have a module that adds something to that screen, you could set the area to permissions, and add the module name to the title. Or vice-versa, depending on how you’ve structured things. This way, the case gets as specific as possible, with the least amount of duplication and clutter.
Summary
The summary is where a lot of good cases fall apart. This is the place to expand on what you started with the title. It should be short, only a few sentences. You want to provide an overview of what is going on, not a novel. This is the place to denote context to the rest of the information you’re going to provide. It should provide a quick overview of what happened, without listing out each step. You don’t need to explain everything in this section.
It should be towards the top, if not the top part of the case, and should be the first thing someone sees and reads when they open a case. Just like the tittle, the body of the case is usually indexed for searching. It’s important to use specific terms and language unique to the case, as it will assist searching.
If you’re unsure how to make the summary clear, think about how you would explain the case to another person on the team. You should be able to clearly convey the meaning of the case in 10–15 seconds by reading this section out loud to a coworker. If it takes longer than that, or if it’s not clear, you most likely need to revamp your summary.
Steps to reproduce
Quite easily the most important section of any average case. Any person should be able to follow these steps, and produce identical results to what the case is describing.
- This section should be a bulleted or numbered list.
- Use short, descriptive sentences for each step.
- Don’t omit common steps. You can combine multiple common steps in the same bullet to save space.
Expected vs Actual Results
This section should be very succinct, and differs slightly from others. It’s a method of internal quality control. This can help weed out cases that aren’t actually bugs, by highlighting what the person creating the case is expecting to see. If this expected description differs from actual requirements, you’ll save development time by catching it before code is changed. This helps keep everyone on the same page when it comes to expected functionality. If there’s any confusion as to what is supposed to happen, this can spark discussion, and help clarify requirements.
Stack trace/error information
Actual data is crucial. Logs, screenshots, databases, or anything related to your application that can be used in debugging should go here. We used to only ask for logs, but found that including various other types of data has increased case visibility and readability immensely.
Screenshots in even the most simple cases can help visualize what can take a paragraph to describe.
We ask that all cases that have a stack trace, that it be included in the body of the case itself. Inline stack traces, copy and pasted within a formatted block, will get indexed for searching. This means I can plug part of a stack trace in to a search, and find similar cases. When logs are attached as files, they aren’t indexed and this isn’t possible. As well, a developer who is looking at the case can see a snippit of what’s happening without loading another application. The goal here is to maximize the ease of finding cases, as well as minimizing the time to fix.
Databases, disk image snapshots, process dumps, Wireshark captures, or other raw data can be provided to help reproduce a specific application or machine state that can be difficult or impossible to set up. Some bugs are data, environment, or network dependent, and loading up the exact setup that was being used to test it will make it easier to reproduce. These additions can help speed up the time to fix a case by a significant margin.
Include the version that this was discovered. Some bug trackers have a field for “Milestone”. I tend to see the milestone field as what version the bug will be fixed, not the version it was discovered. You want to make sure that both are clearly denoted.
Supplemental Information
Finally, I like having a section at the bottom for the leftover information. Not all cases need this, but it’s crucial to some.
- History and Context are important.
- If there is a story behind this bug, make sure to tell it.
- If this is for an angry client, this can be noted here.
- If you need a particular setup or machine, this is the place to say why.
These aren’t hard and fast rules, more like guidelines. Some of them might not apply to your shop, and you might have things to add. The important thing is to establish a consistent format that your people can follow, and then enforcing it. Don’t let laziness be an excuse for bad bugs.