Bug attack — let’s find them all!
I think that most people working in IT know the stress and anxiety that we often experience when The (famous) Release is showing up on the horizon. We start checking everything, adding last tweaks or sometimes even features and making sure that there are no major bugs. In all that chaos and stress it’s very easy to make mistakes or simply forget things. There are many ways to minimise risks connected to releasing our product. It’s very important to try different approaches to make sure that we find the ones that are most suited for the needs of our team, product and customer. In this article I would like to share one of them with you. So let’s make some space for a Bug Bash that in my experience proved to be an invaluable tool in avoiding difficult situations days before The Release.
What is a Bug Bash?
Bug Bash is a meeting taking place some time before your release, where a certain group of people meet and thoroughly test the product in a limited period of time. The main goal of the Bug Bash is to put a lot of stress on the product from every side possible.
In software development, a bug bash is a procedure where all the developers, testers, program managers, usability researchers, designers, documentation folks, and even sometimes marketing people, put aside their regular day-to-day duties and “pound on the product” — that is, each exercises the product in every way they can think of. Because each person will use the product in slightly different (or very different) ways, and the product is getting a great deal of use in a short amount of time, this approach may reveal bugs relatively quickly.
Ron Patton (2001). Software Testing.
Why should you do it?
Why the bug bash session can bring you a completely different perspective to testing your product? Let me mention just a few reasons:
- Thanks to the diversity of the group testing the product, you can view it from different perspectives.
- You get to put additional stress on the app and have the ability to check the performance.
- Thanks to many people testing at the same time you get to check interactions between users within the product.
- Any last-minute or critical issues are caught earlier by the internal team, not the real users.
- You can gather feedback regarding UX and other problems that might detract potential users from using the app.
- The additional benefit of testing for a limited time is pressure that works wonders for creating situations where bugs show up (edge cases etc.)
How to organize one for your product?
Timing
Preferably before any major releases however you should adjust it to the frequency of your releases or specific needs your team might have — remember that you don’t need to have a ready product to do the Bug Bash, you can do one every sprint or only when you fully release certain parts of the product eg. admin panel.
Space
In the current 2020/2021 the most probable location is online, however it’s definitely good to create a time slot and online space where people can be together and interact with each other especially during the presentation part.
Participants
The essential would be to have the whole product team (developers, POs, SMs, designers, SREs).
From my experience, however it is also beneficial to invite people from other departments not directly connected to the product. It will help to imitate the behaviours of potential users that are not familiar with the product and have to do everything for the first time.
You can also consider inviting your customers or even potential users — that might be able to give you a perspective that you cannot imitate
Preparations
- One board/excel sheet shared between users with editor rights — ideally it will serve as one source of truth both for the Bug Bash participants (so they don’t need to waste time jumping from Jira board or between different documents) as well as you — the organizer — when you need to sort the findings afterwards.
- Accesses to testing environments including links, accounts and credentials to all parts of the product that will be used.
- In case there was a recent deploy or some bug fixes, make sure that the product works before the session.
- Presentation describing the product and all the essentials that your participants will need to understand.
- Testing scenarios, unless you want users to choose their own path in the product, should also be prepared and distributed earlier, so our testers can get familiar with them and ask questions or even challenge some of them.
Golden rules for every Bug Bash participant:
- Writing down in one shared document every bug or improvement that we catch with screenshots, recordings or descriptions so the team can reproduce it later on.
- No judgement rule when it comes to any issue we find — during the Bug Bash we need to focus on reporting everything. Time for prioritising will come later and will fall for the team to decide.
- Do not try to analyse the path you are taking in the app just go with the flow (unless you are supposed to follow a testing scenario) — it will help in imitating possible user journeys.
- The time for testing is limited and we should stick to that — the bigger the time pressure, the more prone we will be to make mistakes that can happen with real life users.
- Listen to the moderator and do not be afraid to ask questions when you do not understand how to behave during the session.
Production release can be extremely stressful for the whole team and the last days before the big event should be spent polishing and improving minor things and not stressing over critical bugs or crises that can happen. Performing Bug Bash sessions in your team can help minimise all the negative emotions by creating chaos under control and bringing structure to the priorities we should have days before the release.
If you want to read more about organising a Bug Bash here is a list of some sources that helped me a lot before my first event:
