Scenario Testing and Exploratory Testing of Mobile Games

In 2018 apps for smartphones and tablets accounted for 45% of sales, overtaking console (32%) and PC games (23%) in the global gaming market. The mobile gaming market is growing at least 10% annually — something around $7 billion — according to the Global Games Market Report.

Gaming industry is developing rapidly and we at SimbirSoft cannot ignore this, that’s why we concentrate on helping companies develop and test gaming platforms and individual projects. In this article we will tell you about the reasons of choosing the exploratory testing in the majority of our projects. We hope this information will be useful for both game developers and QA specialists. Let’s start.

Great Game = Emotional Game

“Why do people play games?” — Is the first question we ask ourselves while testing computer games. Usually, a game is not used to solve any problems, but to receive great emotions (especially the positive ones).

Talking about a mobile application, the task becomes more complicated. The user should be happy playing short sessions at a convenient time — while waiting for something or being on the move. After clicking the app icon at the smartphone, from the first minutes the user moves into an exciting gaming world, and nothing should spoil this experience.

While testing mobile games, it is impossible to simply create a checklist and follow it strictly. First, every game is about emotions and it is like a huge tangle in which all the functional, visual and narrative components are interrelated. Game designers create complex and exciting mechanics, programmers write multilevel algorithms and scriptwriters write dialogues, while artists and animators visualize the whole worlds. Both large and small bugs could be found in almost any part of the game, and testers must find and fix all of them. Moreover, testers take into account that fact that users can perform unpredictable actions the game creators did not even think about!

That’s What We Begin Our Work With

The checklist is based on several ideas, which we examine the game with. Let’s consider the main tasks of a QA specialist at the start of testing:

• to determine the target audience of the game (depending on the genre and game setting);
• to determine the features of the mechanics (what is a feature, what is a bug, and how everything should work according to the idea of the game designer);
• what platforms and operational systems (OS) have to be tested;
• to determine the localization of bugs;
• to check the quality of animation and visuals.

Thus, at the beginning of testing, the specialist creates a kind of a guidebook that will help him to check the quality of the game. After its creation, he begins to examine the game world carefully, deeply diving in its atmosphere. Every detail is important here, including the audio part of the game, as there are often bugs in sound effects and in the playback of individual themes. It is also very important to pay attention to the textual description of every item and every dialog in the game — the typos or keys forgotten by the developers are rather common.

Types of Testing

We use several types of testing during the QA activities. Let’s consider on the two most commonly used methods.

Scenario Testing

This kind of testing requires pre-prepared test scripts — sets of test cases. That’s why scenario testing may take much more time than exploratory testing.

Features of this approach:

• a clear understanding of which features are covered by tests;
• confidence that all documented test cases will be completed in time;
• quick and easy involvement of the new specialists to the project due to the detailed scenarios.

Exploratory Testing

First of all, do not confuse this type with monkey testing — exploratory testing does not mean “pressing all the buttons in a row”. The exploratory tester puts himself in the place of the player and studies the project sequentially, its functionality. He uses short checklists instead of detailed documentation and keeps his own test cases in his head.

This approach allows you to speed up game testing. If the tester notices that a certain part of the game is stable, he directs his attention to the other areas — the ones that really are under the risk.

Advantages:

• quick start — no need to waste time creating test documentation;
• you should not create many test cases and update scripts every time;
• acting as a player, a QA specialist finds critical bugs much faster.

Let’s consider one of the most popular online collectible card games as an example. The team needs to test the game session. Here are the differences between the checklists for different testing methods.

Scenario testing

1. Click on the highlighted Sinister Strike card.

2. Perform a long press on the Sinister Strike card.

3. Press and pull the highlighted Sinister Strike card to the playing field.

4. Click on the highlighted Argent Squire card.

5. Long press on the Argent Squire card.

6. Press and pull the highlighted Argent Squire card to the playing field.

7. Press and pull the highlighted Argent Squire card to the playing field. Release it.

8. Click on the Perdition’s Blade card.

9. Long press on the Perdition’s Blade card.

10. Press and pull the Perdition’s Blade card.

11. Click on the Shiv card.

12. Long press on the Shiv card.

13. Press and pull the Shiv card.

14. Click on any opponent’s card.

15. Click on the opponent’s icon.

16. Click the END TURN button.

Exploratory testing

1. Win the game.

2. Lose the game.

3. Check the mana increase per turn.

4. Play for different characters.

5. Interact with the environment on the playing field.

6. Interaction with friends.

This example clearly shows that the scenario approach uses a longer checklist and it has more details. This approach describes the elements and the interaction between them, that’s why its creation takes much longer, but the risks that the tester will miss a bug are lower.

An exploratory checklist describes features or user scenarios. A QA specialist determines himself how much the feature needs to be tested, what and how it interacts with. First of all he passes the critical scenarios and that helps to detect and localize the problem much quicker.

It is important to remember that exploratory testing is for the experienced QA professionals. If the team does not have an extensive experience, the scenario testing will be the better choice.

Other Important Types of Testing

In addition to the methods described, the following ones are also used in testing:

• Functional testing. Its purpose is to identify all possible user behavioral scenarios and test them within the product.
• Performance testing. Its purpose is to determine how does the product influence the gadget, whether there are problems with optimization or not.
• Security testing. Purpose is to check for loopholes in the code that dishonest players (cheaters) can take advantage of.
• Testing the stability of network algorithms. Purpose is to check the performance of the game under the condition of different connections to the wireless Internet, the load on the server, synchronization of updates.
• Usability. Purpose: UI/UX optimization within the framework of constant use and an objective assessment of convenience and readability/readability.
• OS compatibility. Architectural features of certain operating systems affect all of the above factors. Therefore, builds need to be checked everywhere.

From Our Practice

We test a great variety of mobile games, within different audiences. We always try to combine both of the above approaches, for example, we begin the exploratory testing creating small checklists. The article outlines only general aspects of testing mobile games and in different situations the specialist chooses a strategy depending on the requirements of the developer and other factors.

Mobile game testing is one of the most difficult and fascinating tasks at the same time. After all, you should not only ensure the product’s performance, but become a newcomer in the new fantasy world created by developers and make the journey great for others.

Learn more about our QA practices from QA Journal: Risk-Based Testing

Not sure about working with the remote team of developers? Learn about 7 most common IT outsourcing issues in our special article 7 Sins of IT Outsourcing

Follow us on Medium! Never miss an article!