Tester-Free or Bug-Free? It’s Always Your Choice.

By Alexander Kuvshinov, Quality Assurance Engineer at Rosberry

In May 2020 Rosberry celebrated its tenth anniversary. For a fairly young mobile app design and development industry 10 years of work is a pretty solid term. Of course, over this time we have suffered a certain amount of bumps and bruises, but have also built a good knowledge base and collected piles of analytics, which we love sharing from time to time with the readers of our publication.

This time we’ve decided to pay special attention to the issue we often face with our clients when signing a contract. The question is always asked differently, but its essence can be worded in the following way: “Is it possible to do without professional testing and check the application for quality all by yourself?” More often than not such type of questions arise when the budget for project implementation is tight and from the outside looking in testing might seem to be the service you can give up on without much damage to the final product.

To understand if this can be the case, let’s first define what testing is and what tasks a QA specialist usually solves when a mobile application is developed.

International Organization for Standardization (ISO) suggests that testing is the process of research and software check aiming to find out if the actual behaviour of the software (in our case a mobile application) matches the expected behaviour using a set of specially selected tests (ISO/IEC TR 19759:2005).

At the first glance the definition does not seem to entail any complex procedures — you install the application, compare the actual and expected behaviour using a pre-selected set of tests, and, ta da, the product is ready to be released.

But what kind of tests will there be? What is supposed to be the expected behaviour? What exactly do you have to test and check to call your app a high-quality one?

To eliminate possible difficulties of future users to the max, a professional QA specialist in most cases resorts to a whole range of actions and operations. Let’s say we are developing a native app for iOS and Android. One would think that these are just two platforms that require testing. But it’s not that simple. Operating systems (platforms) are updated at least once a year which implies a certain number of versions that can be considered outdated, but would still work on millions of devices, and therefore would require support and testing. What this means is that to develop and test an app we have to deal with not just 2 versions, but with a whole number of them, especially given that many manufacturers (Samsung, Xiaomi, Huawei, etc.) often customize user interfaces for their devices.

Moreover the competition in the smartphone market is forcing device manufacturers to offer customers more and more models with new features and functionalities introduced: biometric authentication instead of a password, three cameras instead of one, contactless payments and lots of others. All these innovations can not but affect mobile applications, so the company engaged in mobile development should always have an extensive fleet of real devices supplemented and updated on a regular basis. Many will say, but there are many cloud services that offer testing with the so-called emulators, i.e. programs that mimic the hardware and software of the target device on your computer. Yes, they help a lot, but unfortunately can not 100% replace real devices.

Or let’s take an elementary functionality when the choice of the country in the app settings determines the content in the app itself.

What can be checked here from a user’s perspective?

• Select a country from the list of countries.
• Make sure that the content corresponds to the selected country.
• That’s it.

And what can actually be hidden behind this simple action? What tests should be carried out? The QA specialist will at least have to:

• Check the boundary values, i.e. do not select anything, select multiple countries, maximum number of countries.
• Reverse check, i.e. deselect and check for changes to the content.
• Make sure that the data received from the server is correct.
• Make sure that the content in different languages is displayed correctly.
• Understand that the application is functioning with no Internet connection.
• See how the application behaves when forced to shut down.
• Check how the app looks on different displays and works with different versions of operating systems.
• Simulate the behaviour of different users, different target audiences, check against negative scenarios, use tricky actions and different testing conditions.
• Test the function of changing the screen orientation when rotating the device.

Of course, that is just one simple example of testing one small functionality. In most cases, the mobile application passes several types of testing at once which in its turn guarantees an appropriate level of quality of the final product:

Functional testing is a kind of black-box testing that is performed to confirm that the functionality of an application or system is behaving as expected.

Change-related testing — validation of the changes implemented as to their influence on the main app functions.

Non-functional testing — testing of performance, security, usability, and design.

Regression testing is a type of testing that is performed at the very end of the project, or during a major update of a certain version of the application. This is about testing the entire application from start to finish.

Actually, using all these types of testing is quite a difficult task for an untrained person.

Moreover, for the testing process to be predictable and manageable, it is also necessary to prepare the appropriate infrastructure and documentation in advance. For example at Rosberry quality assurance starts immediately after the design phase is over. The QA engineer builds the quality assurance process as a system starting with BDD scenarios.

BDD scenario is a written description of the product’s behaviour from one or more users’ perspectives. Each scenario allows even the most untrained person to understand how a user will interact with the application. The scenarios cover all possible user behaviours. This is a clear and a detailed document that together with the design layouts gives a complete picture of the product being created. BDD scenarios at the development stage become the basis for testing of application.

Yet, again it turns out that the preparation for testing begins much earlier than the customer has the time to notice it. And this type of work also requires appropriate training and knowledge.

Bottomline

So, whether you will go with or without a QA specialist is always your choice. But we believe that if there is a QA specialist within your project, you will have much more confidence in the high quality of the product. Development without a professional tester can be compared to a walk through the forest blindfolded running into trees, bumps, branches and what not. You can go forward but it is highly likely you will stumble and fall. A QA specialist will remove the blindfold and clear the way. A tester is like a special-purpose tool, say a cheese knife. Of course, you can use a knife for fish, a knife for meat, bread, or anything to slice cheese. But it is a special knife that will make slicing more convenient and faster. Each item in the kitchen has its own purpose, right? The same with a QA specialist within your project. The cheese is your app, a QA specialist is a sharp cheese knife.