Effective Bug Reporting for Startups
If you’ve ever worked in a startup tech team, you’ve probably had to deal with the onslaught of bugs that’s part and parcel of startup life. Young companies build products quickly. Great companies build them even faster.
That’s a huge advantage when it comes to innovation, and especially when the aim of the game is to reach that sweet spot of product-market fit before the runway runs out, but it’s also a recipe for a tasty bug stew. And if you’re not careful, bug stew is all you’ll be noshing on.
This post is aimed at founders of startups and is approachable for both non-technical and technical team members. I’ll talk about some of the issues I’ve encountered whilst working in small teams and the golden rule for tackling bugs quickly and effectively.
Believe it or not, it’s surprisingly simple to handle bugs effectively. Let’s get started…
Rule #1: Use good tools
This might seem like a no-brainer but I often encounter staff in early stage startups using email (yes email!) to report bugs to their tech team. It’s no good using email where things get lost, are not easily accessible by multiple people, and cannot be categorised properly. This is a sure-fire way to lose track of what’s not working and encourages non-technical staff to send things like “X is not working”… well duh, tell me something useful!
If you actually expect these problems to be sorted then the first step is to use a decent tool. I usually recommend my clients to use a dedicated bug tracking tool such as Jira, which is a somewhat convoluted but very powerful piece of kit.
A fully-feature tool like Jira not only allows you to track bugs but helps your team to plan what work they should prioritise in a sprint, track what has and hasn’t been completed, and report on the velocity of your team. It’s an indispensable piece of kit and something every serious development team should be using.
Investing in something like Jira from the get-go would be a good bet, but if you’re a particularly small team or don’t have a technical co-founder you may need to wait until you have a suitably experienced techie on the team before jumping into a more complex solution like this.
However, there’s no excuse for even the smallest of startups to not be using good tools – your developers are no doubt using a VCS as a repository for code (and if not, why the hell not?) and most providers come with a bare-bones bug reporting tool. One of the most popular providers, GitHub, gives theirs away for free.
That being said, you probably don’t want to give your non-technical staff access to a tool where they could do real, accidental damage. For smaller teams a general purpose project management tool such as Basecamp or Trello which is easily accessible and simple to use for everyone goes a long way.
Rule #2: Train your team
This applies not just to your tech team but to everybody who is likely to need to report bugs as part of their day-to-day job. Front-line customer service staff are a good example of this. They will often get very woolly and useless information about problems from customers and will need to get in the habit of digging deeper and finding out what really happened.
A good bug report should contain the following sections:
- A short title that describes the problem: “User gets logged out when proceeding to checkout”.
- The operating environment: “Mobile website on Android 4.4 using the stock browser, connected at an internet café”.
- The expected behaviour: “When a user proceeds to checkout they should only be asked to login if they were previously not logged in”.
- The actual behaviour: “When a user proceeds to checkout they are logged out and asked to login again”.
- Steps to reproduce: “1) Login 2) Add item to cart 3) Proceed to checkout”.
- Other information: You might need to include information such as the URL where the problem occurred, the severity of the bug (is it blocking something or just a minor inconvenience), screenshots or videos of the bug, error messages or codes, and so on.
It’s important to capture both what went wrong (the “actual behaviour”) and well as what the user thought should have happened (the “expected behaviour”). This is in case the user expects the system to do something differently to what it was designed to do; maybe you need to tweak the system, adjust some copy to make the process clearer, or rethink the entire design of that particular part.
Another reason for capturing the expected behaviour is to help the developer to reproduce the bug on their system. If they know what the user was attempting to do they will be able to narrow down their search for the bug. Often it’s not obvious exactly where a bug stems from, so the more information you can provide them the better.
Difficult to find “edge case” bugs are not immediately visible and it may only be a very specific set of circumstances that trigger the bug. Consider our example from above; the user who gets logged out at checkout may have originally logged in with Facebook and later changed their email address on your system.
Now, does the checkout bug stem from when the user registered using Facebook? Or when they changed their password? Or when they clicked ‘proceed to checkout’? Or is it somewhere seemingly unrelated? Well, all that quickly becomes irrelevant if the developer had to spend hours digging through code because the bug report didn’t mention that the user recently changed their email after registering with Facebook. Doh! 🙄
Rule #3: More information is always better
It may be easier for the developers who know the system intimately to guess where bugs are likely to crop up. They know which parts of the system are solid and can be relied upon, and which parts are likely to flake out at the slightest exertion.
It may seem like developers are best suited to writing bug reports and in some (more technical) circumstances that’s certainly true. For their vast majority of situations however, the closer the staff member is to the user, the more suitable they are for bug capture duty. Don’t play a game of Chinese whispers and expect the developer to divine what on earth is going on from cryptic messages.
The key to a good bug report for non-technical staff who have no such prior knowledge, is to add as much information as possible to the report. There’s never a time when adding too much information is a bad thing, as long as that information is relevant.
The following are some questions that are pretty much always relevant when capturing information on bugs. This is by no means an exhaustive list but should be enough to get you started:
- What happened (the actual behaviour)?
- What should have happened (the expected behaviour)?
- Can you walk me through exactly what you did from beginning to end (steps to reproduce)?
- Does it happen every single time?
- If it’s intermittent, when does it occur (what’s the trigger)?
- When did you first notice the problem?
- How many times has the problem occurred since then?
- What type of device are you using (desktop, laptop, phone, tablet, watch, etc)?
- Do you use more than one device?
- If you use multiple devices, does the problem occur on all your devices or just some?
- What have you changed recently?
- What error message and error code is displayed?
Some example of other questions you might ask depending on your setup or situation:
- What operating system are you using (Windows, Mac, iOS, Android, Linux, etc)?
- What browser are you using (IE, Edge, Chrome, Firefox, Opera, Safari, etc)?
- What is your internet connection like?
- Where are you accessing the system from (a work network, free Wi-Fi, at home, etc)?
- Does the problem occur immediately when you click X or after a delay?
There are many more questions that could be asked, of course, but knowing the right questions to ask will be down to your team understanding your product inside and out.
The aim of the game here is to gather enough information so the developer can reproduce the problem on their own system. If they can’t do this, it can often make it economically impossible to track down and fix the bug.
Rule #4: Help the customer immediately
Customers can be fickle and are often impatient. If something goes wrong you should be aiming to fix their specific problem immediately. Granted this isn’t always possible, and can be exacerbated by the fact it may take several hours or days to track down and fix the bugs causing the customer’s woes.
The delay between being notified of a bug and pushing out a fix can seem substantial to a customer who has no insight of the processes and technical challenges within your business. Therefore, it’s important to reduce the time-to-fix by providing alternative solutions.
It’s a no-brainer to train front-line staff in the common and basic fixes that can be tried to eliminate the problem. We all know how effective it can be to turn something on and off again and it’s always worth a try. 🙏 These kind of simple fixes are easy to explain, quick to try, and make the customer feel like something is being done. Of course it goes without saying, don’t waste their time (or yours) with dozens of “quick” fixes if they have little chance of actually working.
Similarly, there is usually always a “manual” work around that front-line staff can do, or that developers can implement more quickly than a full fix. This isn’t to say put a sticking plaster over the problem, brush it under the carpet and hope you never see it again. But it is to say, if you can reset a user’s password for them because the forgot password form is broken, then do it. Don’t make them wait until the bug has been fixed and expect them to still be around to care.
These small fixed and workarounds are also useful in helping the developers narrow down where bugs may be hiding so make sure all your teams communicate what is and isn’t working to fix issues. And more importantly, they give the developers time to figure out a good solid fix rather than a quick and dirty hack, which will save time and money in the long-term.
Another way you can save substantial time was eluded to in the previous section. Save the valuable time of your developers by getting the information they need straight from the horses mouth, when it’s quickest, easiest, and cheapest to do so. That means front-line staff asking the right questions and reporting enough information from the get-go.
Knowing what can and can’t be done to help a customer quickly depends on there being smart lines of communication open between the tech team and the other parts of the business. A solid bug reporting process is one such line of smart communication.
Rule #5: Implement a solid process
Great, so you have some awesome tools in place and a team who know what they’re doing, and yet you’re still drowning in bugs. Problems aren’t getting rectified quickly enough and things are getting lost in the weeds.
As with most things in business; figuring out what works, codifying this into a repeatable process, and ensuring everybody is on the same page is the only thing that will cut the mustard. Solid processes are the bread and butter of successful businesses and bug reporting is no different to anything else.
It’s one thing to tell people what they should be doing, and another thing entirely to ensure they stick to the agreed upon process. Once you have everybody pulling in the same direction, collecting all the required information and using the appropriate tools you’ll be able to move much more quickly towards fixing those bugs.
Don’t forget to ensure that your developers are also filing bug reports correctly. Often, developers can be the worst at reporting bugs, especially when they speculate on where the problem lies rather than sticking to the facts. It’s fine to have a hunch, but know there’s a good chance it could turn out to be wrong. Those bugs are pesky critters and are adept at hiding in very strange places.
If you want to be a world-class product that customers rave about, everybody in your organisation should be concerned about product quality and continuously raising the bar. This is especially true for those on the front-line who deal with customers daily.
However, it doesn’t hurt to have another pair of eyes dedicated to assuring the quality of your product and catching bugs before they’re pushed out to production. Considering hiring a dedicated QA who will help setup a process to reduce the probability of users encountering bugs by managing the control of source code, reviewing code changes, testing, and coordinating releases to meet deadlines, amongst other things. In short they oversee the entire development process.
Hiring a QA is not, however, an excuse to offload the responsibility for writing good code to someone else. Neither is it an excuse to offload the reporting of bugs which should be considered a “universal responsibility” for all staff.
Conclusion
So in sum, bug reporting is a very simple practice, in theory, but one that requires a solid and enforced process, and a strong cultural foundation of continuous improvement to underpin it.
You must find and use good tools for logging, categorising and managing bugs, email will not suffice. Staff training in writing proper bug reports is essential and will save many hundreds of development hours, it’s especially important for front-line staff who will come into contact with bugs more often. Make sure all bug reports include lots of detail, most importantly the steps to reproduce the problem, the expected result and the actual result. And of course help customers immediately to give the developers time to fix the problem properly.
Most importantly, you must turn all of this into a solid and repeatable process, making sure everybody is on the same page and pulling in the same direction. Hiring a QA expert is a good way to do this.
And so this leads us to the golden rule. As with many things in life, it’s stupendously simple but devilishly difficult to get right – you must be disciplined in your approach to bug reporting. Yep, that’s it! Every good process requires discipline to get it right.
Happy bug hunting! 🐛
Thanks for Reading
If you liked this article please recommend it! 💚
About the author
Hey I’m Josh. 😄 I’m a tech entrepreneur currently living it up in London (the UK one!) on a journey of discovery, positivity, and innovation. I have a real passion for technology — I believe technology will help us to solve the biggest and most challenging problems facing humanity today and in the years to come. Call me an eternal optimist. 😁
If you’d like to read more of my content please follow me on Medium and Twitter, and feel free to connect with me on LinkedIn.
