Giving great bug reports

Every software developer has to do their fair share of tech support, and I’m no exception.

Of course no software is perfect, especially if it is solving a complex problem, and sometimes things do go wrong. I also find myself supporting other people’s products in my volunteer capacity as “the one who knows about tech stuff”, so I’m used to emails asking for help with something not working the way it should be.

A bad report

Very often, the first tech support email I get from someone has, as its main text, something along these lines:

“I cannot log in to Careview.”

“Google tells me it cannot find my account.”

“I get an error when I try to use DigitalUrban.”

It’s easy to understand why someone might write an email containing a vague statement like this. The software didn’t behave the way you wanted it to, you don’t understand what went wrong, and you just want someone to fix it for you.

The problem with emails like this is that they don’t contain enough information for your friendly, helpful tech support person to be able to fix things for you.

Below is — hopefully — a handy guide to providing quality bug reports so that your tech support person can get to helping you immediately.

First: reproduction

Before you fire off your email, it’s important to check if your bug can be reproduced. What this means is: “when I do the same exact thing again, do I get the same error?”

Go back and do the same thing you did before and see if the same thing happens.

This is important, because reproducible bugs are much easier to track down and fix than intermittent ones.

If your bug can only be reproduced some of the time, or even if you can never reproduce it again, it’s still worth reporting! But in this situation it’s worth mentioning to your tech support person that it’s intermittent or can’t be reproduced. These kinds of bugs are much harder to fix, so you’ll probably need some back-and-forth emails before the fixer can get to the bottom of it.

For the purpose of this article, let’s assume the bug can be reproduced. These kinds are much more common and usually easier for the developer to understand.

Second: document reproduction steps

OK, so now you’ve proven to yourself that you can reproduce the issue, it’s time to write down the steps you took that led to the error.

Which page did you visit first? What did you click? What did you type, and into what boxes? And so forth.

Even if this all seems really trivial to you, it’s really worth writing it down, because what you may think of as a common way to use the software could in fact be one that the developer has forgotten about or wouldn’t have thought to pay attention to.

(Note: when sharing your reproduction steps, be careful not to divulge any sensitive information. For example, you can just write “I type my password” instead of actually writing what your password is.)

Third: document the error

When the error occurs, it’s important to report exactly what is said on the screen.

There’s a temptation here to paraphrase, but this can be very confusing to the developer. For example, recently someone said to me “it says the page cannot be displayed”, while the message on their screen was “Coming Soon”. From the user’s perspective, they were reporting this accurately (the page they wanted couldn’t be displayed), but it meant I went down a rabbit hole of investigation because “page cannot be displayed” are the exact words from a well known Internet Explorer error.

It can be very helpful to copy-and-paste the text of the error into your email, or get a screenshot. You can take screenshots on most computers and mobile devices, and here is a guide to how to do it on your device.

Finally: include environment information

I’m not talking about how often you recycle or whether you’re burning fossil fuels… the environment is what your system looks like.

Good things to include in your bug report are:

• The make and model of your device, and the operating system it is running, if known
• The browser you are using (Chrome, Firefox, Internet Explorer, Safari…) and the version
• Which URL you were visiting (you’d be surprised how often we misunderstand a bug report as being for a completely different product!)
• Which user you were signed in as when the error occurred — this is especially useful if you sign into the software using a different email address from the one you’re reporting the bug from.

An example

Taking all these into account, here’s an example of a really helpful first email about a bug:

Hi Quinn

I’m getting an error every time I try to log in to Careview.

Here’s how I reproduce it:

1. Visit https://careview.example.com/
2. Click on the heart
3. Fill in my email (bob@example.com) and password and click the “Log in” button
4. I get an error in a pink box saying “Invalid Email or password.” I’ve attached a screenshot so you can see what it looks like.

I’m using a Nexus 6P phone running the latest version of Chrome.

Thanks for your help!

As you can see, they don’t need to be really long and verbose to capture all the information!

The “every time” tells me you can reproduce the issue. The numbered steps show me the reproduction steps and also the text of the error message. The screenshot attachment helps me see what you see, and finally you’ve included your browser and device information. All of this is very helpful, as you’ll see in the next section.

How we help & why these are important

The process of debugging (finding a bug) isn’t an exact science, but a lot of things are very common:

• If we know the error message that’s been reported, maybe we can explain the issue immediately (or hike the priority of fixing it, now it affects a real user).
• If we don’t know the message, but we have the exact wording, we can search the codebase for it, which saves an enormous amount of time. This doesn’t work with paraphrases so the exact wording of the error message is very important.
• We might try following the reproduction steps with our own accounts or with a test account on a development environment with debugging tools enabled. Many Fish Percolator apps are written in Rails, which has extremely good debugging tools when running in development mode.
• Maybe we’ll try and get hold of a device like yours to debug device-specific issues.
• We can look through the server logs, filtering by your user information, to try to see the error from the software’s perspective.

All these things and more are only possible if the email contains a good bug report: hopefully this article has helped you get inside our heads ready for the next time you get frustrated with software not behaving itself!

Edit: See also this great advice from John B over on Twitter: