The Art of Exploratory Testing

What comes to your mind when you think about software testing? To many people, it can be an endless list of formal test cases that takes a long time to execute. However, there are many approaches to testing. On the other spectrum of very structured testing lies the art of Exploratory Testing, which emphasises on free-play and thinking on the fly.

Source: Unsplash

Exploratory Testing is a type of software testing whereby testers check software applications based on their experiences, by exploring, discovering and strategising the tests as they go along.

Why do Exploratory Testing?

Exploratory Testing is the best way to discover edge cases that aren’t normally caught by structured test approach(es). A skilled tester can quickly come out with peculiar flows and unique combinations that have a high probability of being handled incorrectly. An experienced tester will, also, know the common mistakes developers make and zoom into those tests that check them.

Feedback can be provided faster without needing to run through test cases with possibly many steps to execute. Potentially, especially when the tests are smartly focused on problematic areas, more errors will get discovered in a shorter time. This is especially useful for time-critical projects or teams that need to quickly roll out a Minimum Viable Product (MVP).

One strategy in which exploratory testers can use is to put themselves into the shoes of different user personas, such as an elderly or a young student. This provides perspectives from different types of users and may uncover actions or flows that are out of the normal pattern of a typical user.

Here’s a table of some generic tests to try on the application in the exploratory process to uncover common unhandled issues:

Crowd Exploratory Testing

So what is better than a tester doing Exploratory Testing? A group of testers doing Exploratory Testing of course!

Crowd Exploratory Testing is not an official term but it’s what we’ve practised in our teams from time to time and proven useful to gather valuable feedback from a group of people. In short, it is a group testing session of a product by a team, volunteers, or even the entire department. In a time-boxed period, the group will perform Exploratory Testing and give their feedback to the product team members, who will follow up with the highlighted issues. This gives an even broader perspective from various points of view of people of different roles, and also an opportunity to cover different devices and platforms owned by different people in the group.

As useful as it is, the product team conducting the Crowd Exploratory Testing needs to plan and manage the session properly in order to document all feedback and follow-ups effectively, without turning it into one big mess, especially with many repeated observations and clarifications flying around. Good facilitators and reporting tools/processes certainly help. In our case, we usually create a communication channel for participants to provide screenshots with their observations in a predefined format so that team members can refer to easily.

Any downside?

Having said so much, how about the downsides of Exploratory Testing? The most obvious one is that it depends a lot on the tester skills, experiences, creativity, and may require good domain knowledge of the application under test.

Lack of documentation may also be resulted due to the free nature of Exploratory Testing. The last thing we want is to find a bug and forget the steps to replicate it. However, there are many tools available to aid the recording of steps and screenshots. One such free tool is the Chrome Exploratory Testing Extension.

Even after doing Exploratory Testing, structured test cases are still needed to sniff out basic issues that could easily be missed or hidden among a vast amount of features. This can be achieved efficiently using automated regression tests to repeat the mundane and often repetitive checks.

Based on the timeline and criticality of the application under test, Exploratory Testing can be considered to be part of the testing strategy, and can often be the first test approach of a new feature to provide a fast feedback loop. Feel free to share with us about your experiences in Exploratory Testing too!

🧙🏼‍♀ Team Merlin 💛
Application security is not any individual’s problem but a shared responsibility.