What’s Exploratory Testing?
and Why Should we Use It?
Let’s face it: End users often don’t use software / apps / systems the way that developers expect them to. Here are a couple examples from my own experience:
- We all probably have that relative or neighbor who creates flyers (or <shudder> their holiday newsletter) using Excel because they know that it will allow them to easily format text and images neatly.
- I once tested a screen for an ERP system and completed the fields out of sequence (that is, in a different order than from top to bottom) and caused the app to crash every time I clicked the Save button.
Humans will use software in ways that work for them, not necessarily in the ways that they were designed or the ways that those of us in the engineering department thought they would. Therefore it’s important for QA testers to spend some time using the software like a user would. This is where Exploratory Testing is extremely helpful.
Many of us are familiar with scripted testing:
- Login to the system
- Navigate to the user profile
- Enter “123 Main St., Anytown, PA 12345” as the Home Address
- Click the Save button
- You should receive a “Save successful” message
Scripted tests are great for ensuring that the app is working according to the specifications (that is, “fitness” as described in my earlier post on the definition of quality). Almost all automated tests are based on scripts that are executed by the test framework. (“Almost all” because some AI tools are starting to test software/sites without a script.)
Stated simply, Exploratory Testing is the “opposite” of scripted testing: The tester frees their mind (as much as they can) from their scripts and the written requirements and just uses the system.
One of the handiest ways that I have found for helping testers do Exploratory Testing is with a Test Charter. In most charters, we set a goal/outcome or declare what user path to explore, set a timer, and allow the tester to attempt to accomplish the goal or finish the user path within a time limit. As a practical example:
To test our site’s in preparation for Black Friday, assume the persona of a busy parent who has budgeted $300 for gifts from our site for their three children ages 2, 7, and 9. They are trying to complete this task during their 30-minute lunch.
Note that in this example, we don’t tell the tester what to test and how. We don’t define that they must use the Search functionality or to go to a specific starting page (for example, our Black Friday Specials), and we don’t even tell them what products we want them to add to the shopping cart. They have an end goal (buy $300 or less of holiday gifts for three children) and a time limit (30 minutes or less), and the tester is allowed to figure out how to accomplish that goal using our site. I know testers: Some will use the Search. Some will see “Black Friday” in the charter and visit that page. All of them will view our product display pages for various items, click our “add to cart” buttons, and move through our shopping cart user flows. I also know that, if there are major bugs, they will be found easily by these testers. I know this because good research has been done showing how effective Exploratory Testing is at finding bugs.
One of my other favorite use cases for Exploratory Testing is to learn about a new system or feature.
- We just revamped our Search functionality? Cool — I’ll spend some time doing all sorts of silly and serious experiments: null input and click the Search button; a nonsense word like “foofle” to see what errors I receive; clicking the Search button three or four times quickly to see what happens with an unusual user interaction. I know that I will both learn about the function, and I will find some bugs if some of these unusual interactions haven’t been anticipated by the developer.
- We just added the ability to store and use credit cards for our eCommerce site? Great — I’ll maneuver around test cards, invalid information, processing both valid and invalid payments, etc.
Now that I’ve sung its praises, I also need to note the biggest drawback of exploratory testing: It is extremely time consuming. In my example charter above, we’re allowing the tester to spend up to 30 minutes. If they encounter a bug, they should suspend their timer for the time needed to write a good bug ticket, then come back to complete the task if they can. We can run a lot of automated tests — and even a few scripted manual tests — in those 30 or more minutes. As a test manager, I want to work with the team to determine an appropriate cadence for long-running exploratory testing since it diverts the testers from other important types of testing.
When used well, Exploratory Testing is a great tool in the QA Team’s toolkit. It helps us to learn about features and systems, and it can help us check for usability concerns or issues. It also usually results in more bug reports than scripted testing would. I highly recommend ensuring that your team uses some amount of Exploratory Testing!
Hunter Strategy uses a combination of approaches when it comes to testing software, including exploratory testing. Contact us to learn more about how our software engineering and product development solutions can help you!
Contact Us
contact@hunterstrategy.net
