Go to the profile of Emir Gocen
Emir Gocen
PMP & PSM I certified project manager specialized in mobile application world.

Benefits of exploratory test before pilot

If you have never heard of the term “exploratory testing” before, it is an approach first coined by Cem Kaner in 1984 by this book where the professor states it as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester con continually optimize the value of her work”. In here firstly the tester goes through a learning cycle; anything that guides on what to test, how to test or how to recognize a problem is beneficial. Then a plan on how to test is designed, however, test scenarios are not designed. Exploratory test is executed and results are collected, this can be manuel or automatic. Lastly, the results of the test are interpreted to learn maximum for the state of the product and the bugs.

The benefits of exploratory test are:

  • Takes less preparation,
  • Critical facts are found quickly,
  • The results of previous tests can be used to reason out future tests,
  • Rapid feedback can be provided in limited time,
  • Suitable if requirements and specifications are incomplete,
  • Helpful for dynamic products which can be unstable.

A good and short video can be found on this link regarding the benefits of this methodology again.

Its drawbacks are:

  • Tests are conducted on the fly without a script so it is hard to show how a bug is reproduced,
  • Tests can not be reviewed afterwards so it is hard to keep track what are tested,

When you are a startup that has reached an initial state in which you can move on with scripted tests, when there is a need for compliance testing (meaning you must adhere checklists, regulations, government rules, etc.) or when there are areas to test with rigor and close monitoring you should not be using exploratory testing as written in here.

I want to point out a different usage for exploratory testing: conducting it just before pilot. For multiple projects we have done this, it was very useful to get feedback from potential endusers and to inspect bugs or UX/UI misuses which the development team might become accustomed thoughout the project, however, new sharp eyes can detect.

How it can done:

  • +10 users from different departments are gathered in a meeting room,
  • These users can be quality assurance (QA) engineers but not just restricted to them,
  • It is recommended that these users are not part of the development team but rather people who are seeing the product for the first time,
  • Provide test devices to these users, in accordance with the device and OS distribution of the app to the market you are planning to release,
  • Provide white blank sheet and pencil/pen to take notes,
  • Try not to provide information on how to use the app or test cases and let the users discover the app,
  • Ask users to take snapshots for possible bugs,
  • Talk simultaneously with the developers is you are unsure about a screen/feature while testing,
  • Have at least 2 QA engineers from the development team to monitor the exploratory testing and answer questions,
  • Make several sessions if there are a lot of attendees,
  • Once the testing sessions are over, create a report on the process with the bugs found, the positive and negative feedback, actions to take and share it with the business or customer.

Best.

Emir