How bug bashing frequently and consistently changed the team mindset towards quality and built connection at QuintoAndar.
This post talks about how we used the bug bash technique at QuintoAndar to deal with the challenges regarding quality assurance in order to become a stronger and more integrated squad. To get more details on how to create a bug bash, check our step by step in the article “A guide to starting bug bashing now”.
Introduction
Bug bash methodology has been used by many companies in the past, especially those working on an agile development environment. Due to its integrative trait, the Bug bash yields a broader spectrum on test coverage provides a deeper understanding of the product for all team members and even improves the development cycle speed. This testing approach has been applied to different squads at QuintoAndar, becoming a popular practice to assure both the product’s and team’s Quality.
The challenge
So I am the only honored quality assurance analyst working on a multidisciplinary squad with an agile approach to development — and this is the default setup for many peers on the field, generally speaking.
There are 4+ developers working on a different feature each, targeting releases at least twice a week. How the heck do I keep up with the testing routine without working myself to death or going completely nuts? We needed a better test strategy! We already had a Regression Test suite to ensure that our existing product was still fully functional and we also had a smoke test suite to run before any hot-fix go live. However, we were a bit uncertain about how to document and execute tests for new features in such a short time. That was when I recall a concept called Bug bash.
Although sparsely, the technique was already being applied at QuintoAndar The QA team in collaboration with developers, has put effort into leading bug bashes before big releases. Despite that, we lacked frequency and organization to turn that into a practice that would benefit the product. Moreover, we also fall short on sharing quality assurance responsibilities. Thinking of that we asked ourselves: could bug bash also bring awareness about the importance of quality assurance — or what we called quality mindset?
The challenge of getting to know how to build a process to apply the bug bash effectively in different contexts is a whole other story (that you can check on “A guide to starting bug bashing now” and “Product Design & bug bash: how to validate interfaces and quality assure your delivery from end-to-end”). Right now we want to focus on what we came to believe is a bigger challenge: a quality mindset!
The QA’s role
When you hear QA, you might be talking about several different roles such as QA engineer, QA analyst, QA tester, QA automation engineer and so on as described in the article “The different roles within the field of QA and testing”. Despite the differences among these roles variations, the Quality Assurance person is someone focused on the final product, bringing the end user’s perspective closer to the development, yet keeping awareness of the technical limitations on different areas. A QA will always be on the intersection between Development, Product definition and Design, making sure the best product is being built given the circumstances. Working on this hybrid setup means the QA’s inputs are especially valuable in the initial phases of development, where frequent problems are mapped, hypotheses are validated, doubts are raised and ideas are consolidated into action points.
Therefore, to make the most of the QA’s skills, one needs to be included and to participate in the entire creation process: from the brain-storming to the user story-mapping all the way through the feature launch. Yet, usually, the QA is more actively involved later on in the project becoming, more often than not, isolated from the team, working on a silo, tackling tests alone, being fully responsible for the feature’s validation. Generally speaking, in the software development field this usually happens because the team believes the QA phase starts once the code is implemented. Besides, historically, teams assign the testing to the QA analyst solely.
That mindset can be frustrating for the QA, not to mention a very unhealthy and risky practice for the company. Because one person looking to the final validation, represents…well, one person! And that means a single person’s biased perspective projected into the product.
However, this is such a common reality on the QA field that even though the Dev-Qa ratio is often very unbalanced, the QA tends to feel very compelled to get all the tests creation/execution and quality assurance responsibility to oneself. Afterall, QAs are the “quality guardians” of the team, right?
Well, yes. And, no.
Yes, the QA should have the last word on releasing a feature or not, based on the severity of the problems found. Yes, a QA should be aware of all the possible impact areas of this new feature on the product. Yes, QAs should do my best on implementing processes that allow me to ensure test coverage.
And, no, the QA is not the only responsible for, or capable of, executing all possible scenarios to ensure the product is thoroughly covered by manual and automated tests. Indeed, because of the hybrid character of the role, the QA is always trying to forecast a wide variety of use cases and might have more context or knowledge about different pieces of the project. Still, that does not mean one knows it all or there is nothing else to be added to that test plan. And even if one wanted very badly to do it by oneself, it would not be humanly possible because, usually, there are just too many ways of using a product.
Is there a better way, though?
I am positive that are plenty of alternatives to be explored out there for this soon to be outdated QA reality. Nonetheless, at QuintoAndar my squad’s response to this challenge was what we called the quality mindset.
I was involved in the initial phases of each project bringing more than my test skills to the table while all developers, product managers and designers were involved in the testing phase bringing their priceless insights and customized use on the product. Using the bug bash approach we began to engage the entire team on the test phase without compromising any bodies’ agendas since it is a relatively fast ritual, carried days before launch. That in itself gave me room to breathe and be more active in the initial phases of the projects increasing my morale and my contribution to the company.
This also meant that not only me, the QA analyst, provided inputs on how the final user could see the product; each individual of the team would bring their unique perspective, built from different backgrounds and experiences into the final deliverable. They started to have the time and space to try the initial idea in a practical fashion, turning the whole validation process into an opportunity to exchange knowledge and gather feedback. For most of the developers, it meant to understand the product holistically. For product managers and stakeholders, it meant to go deeper on the technical aspects.
What about numbers?
Talking results, the impact on different types of deliverables speaks for itself. In general, we tend to raise a lot of questions and doubts although, at the end of the day, only a portion of the findings are reported as real problems and only a small percentage is considered as show stoppers. Nonetheless, this shows how much the process contributes to the quality assurance of the features, regardless of their nature. This also means strong alignment among team members, who decide the priority of the issues together.
At last, the whole team is looking to guarantee that a major variety of scenarios are being covered before launch. There are no more silos, we are all equal owners of a built-together product from top to bottom.
Sounds like magic, right? The truth is that it is a constant exercise and will ever be. There are no “one-size fits all” in agile software development. Sometimes it works perfectly, no reservations. Other times, so many things go wrong that are mind-boggling. The secret is to keep iterating and improving, always adapting the process to the current context. The key here is that now we are in the same room, bonding, and building trust. We are no longer talking about a quality check.
We are talking quality mindset.
