What Skills are Required for a QA Engineer at a Product-based Company?

Do in-house and outsourced QA engineers need the same skills? I’ve been working as Head of QA Department at AIBY Group, a product-based company developing mobile apps, for years. Having analyzed my experience, I’ve prepared this article to highlight the difference between them. Let’s dip into the world of testing.

I’d like to point out that, usually, outsourcing implies hiring QAs to do specific tasks. Those can be:

• Manual testing, which is frequently just functional testing according to given requirements, interface testing (the comparison of the given interface to the references, without any impact on UX), and localization testing
• Automated or load testing

There are exceptions to this, of course, but most outsourced projects boil down to these tasks. Meanwhile, an in-house QA isn’t limited this way and is more deeply involved in every stage of development, having a significant an impact on the product.

Next, I’ll go through the main stages of app development to cover the skills a QA needs at each stage.

Click on the image to enlarge it.

Ideation in product development

Although a QA doesn’t generate key business ideas, they can offer suggestions which may impact the product. Such ideas must be timely and respond to business needs. For instance, there’s no point brainstorming ideas which don’t correspond to current objectives but overwhelm the backlog and team. For the ideas to be relevant, a QA has to understand how users interact with the app and what their interests are. A good grasp of company priorities also matters.

This way, product QAs

• should be able to prioritize current objectives and understand Product Vision (e. g. ideas during MVP launch and the ones generated while retaining users will be drastically different).
• need to have a good grasp of product metrics and methods of testing hypotheses.
• have to understand the principles and methods of usability testing and be ready to test and give questionnaires to users.
• should know their way around marketing and user stats tools.

Communication with Product Managers, the analysis of metrics and stats, and learning how to use the above-mentioned tools will help to improve these skills.

Working with requirements

Many product companies, especially in the startup stage, do not hire business analysts who would typically be responsible for collecting and preparing requirements. It should be noted that product managers are not business analysts. They are the drivers who keep generating new ideas while the team is still working on the previous ones. Product managers may simply lack the time or technical expertise (no offense to anyone in particular, but it really happens) to prepare detailed requirements in the proper form.

This is a situation when you need a person who will deal with unspecified or contradictory requirements. This function should be split among everyone involved in the development process. My experience shows that a QA usually does this work with passion.

“Filtering” can be done while the requirements are being prepared or when it’s time to move on to new feature testing. The second option is more common, but the first one is more effective and so preferable. Unfortunately, when a QA starts testing the requirements before the development phase, they are often asked not to review, check, or proofread anything yet because “it isn’t ready”. I’ve repeatedly faced situations where the “it’s ready” phase actually revealed that the previously raised questions had been relevant, and if they had been responded to in time, there would have been fewer edits later.

Outsourced projects may often fail to do requirements-based testing due to the limited budget, deadlines, and scope of responsibilities. Product companies normally do not have such limitations, as the requirements are prepared by the team, so it is extremely important to set up early requirements-based testing and make it clear why the questions raised are significant.

On the other hand, inexperienced QAs often insist on outlining all requirements in an early-stage product. This is not particularly effective: the product is changeable, and focusing solely on the prescribed requirements may slow down the development process.

That’s why it’s important to build a flexible process for working with requirements. The following may be helpful:

1. Experience with a variety of processes and technical education which should include workshops on the preparation and management of requirements. Another option is training in the fields of business analysis or working with requirements.
2. Methods of argumentation, communication, and objection handling. They can be developed by means of special training or personal growth.

UI design and UX

When releases become focused on user retention, the most essential parameter of product quality is usability — a clear interface, logical navigation, and attractiveness. It is necessary to set up two processes: usability testing during the prototype and/or design phase and the same during the main testing cycle. These processes are mandatory in a product company, but they are often absent in an outsourced project.

By the way, usability testing is the basic competence of a manual QA (if we refer to the ISTQB certification). Therefore, making suggestions with regard to prototypes and design is quite reasonable. 🙂

Basic expertise needed for usability testing:

1. General principles of interface design and usability;
2. Perception of color and iconography;
3. Platform (iOS, Android) guidelines on UI elements and technologies;
4. Working with texts, which includes assessing the grammatical correctness and informational content;
5. An ability to intuitively understand and feel users (how they use the application, what mistakes they may make) as well as put yourself in their place;
6. Understanding of the peculiarities and preferences of users from various countries.

Development

In this stage, there will be no difference between product companies and outsourced projects.

Testing specific features requires algorithmic thinking — that is, understanding how a feature is programmed and predicting possible bugs. Regardless of whether you have access to the code, you need to have a basic understanding of what algorithms have been used. In this connection, having an education in the field of software development (either a university degree or training course) might prove useful. It will help you better understand developers as well as decide which tests to run and which not to.

Testing

Testing features that affect monetization — ads, in-app purchases, promotional and landing banners, the collection of statistics — is a priority in product testing and its main distinguishing feature. For each monetization method, there are certain implementation and documentation guidelines, specifics of testing, and tools.

Some testing types might not be mentioned in the requirements documentation ​​ and aren’t usually discussed, yet they shouldn’t be omitted.

Let’s look at some examples.

Compatibility testing

It’s important to ensure that user experience with various devices, OS, and browsers isn’t interrupted by logical, interface, and performance bugs.

Mistakes to avoid:

• Testing on one device (high risk of missing some bugs);
• Testing on all devices (too time-consuming)
• Using only the most popular devices (they might have identical technical characteristics, which results in ineffective testing)

In outsourced testing, configuration parameters are limited or set by the customer. When it comes to product testing, you have to define configuration parameters on your own. You also need to have all necessary devices. Here are some things to keep in mind:

1. You have to be familiar with different device brands, models, and distinctive features (screen varieties, processors, graphic coprocessors, camera parameters, etc.);
2. It’s important to know how different OS versions and settings affect the application behavior;
3. You should keep up to speed with the latest tech development news and timely update your device park (new devices and OS versions are released several times a year).

You can’t gain this knowledge at university, and nobody clearly talks about it. But you’re supposed to know these things, so you should find the time and resources to learn them on your own.

Client–server application testing

Things to know when testing client–server applications:

1. What a backend API is and how to test it;
2. Which client–server protocol is used and its specification;
3. Which tests to run on the server side and which on the client one; which tests should be performed on both of them;
4. Database structure (for example, a change in database structure involves data migration, which leads to a higher risk of critical bugs);
5. Tools for sniffing traffic and database management (information retrieval, generating test data with SQL queries).

While this is also important in outsourced testing, there’s a great possibility of strict limitations being set by the customer (for example, having no access to a database), the division of responsibilities (back-end developers test API on their own), and the inability to use tools. When it comes to product testing, there are no such limitations. The more the product develops, the more important it is to understand every tech aspect. You either gain such tech awareness at university or at extensive courses you take while working (back-end development, databases, etc.).

Localization Testing

Sooner or later, product companies start developing apps worldwide. Obviously, the world is huge and full of people with different mentalities, languages, and habits. So, checking whether there are translations into selected languages is one thing, while analyzing their adaptation to users from target countries is a different story.

A well-localized app must be adaptable to certain regional parameters (date and time, scale of temperature, units of measurement, currency, calendar type, etc.) and to the peculiarities of people’s mentality (the perception of colors and symbols, sense of humor, and much more). Every country has its own holidays and favorite characters, so app content creation may be based on this info. To check the localization for a certain country, one should either get acquainted with the country’s national characteristics on their own or hire native QAs. However, the latter is likely to add certain complexities to the process, as additional hiring and performance control tasks arise.

The same is true for load testing, security testing, and other technical directions which every product sooner or later takes. A product QA should be able to prevent bugs on time and organize the relevant testing process. In outsourcing, the customer usually decides whether these types of testing are necessary.

Support and user post-release behavior analytics

The Customer Support team often redirects either bug notifications or user questions to QAs:

• To find the root causes of bugs, you need the skills mentioned above. In many ways, it’s similar to outsourcing.
• As for the incoming questions, they give relevant information about the improvements. The App Store and Google Play reviews play the same role.

Product QAs often have access to app statistics. It helps them analyze user cases, the duration of user sessions, types and amounts of processed data, the most-used configurations, and the errors customers frequently encounter. With the help of this info, QAs improve test scenarios, make the testing process more productive, and, very importantly, localize user problems as quickly and accurately as possible (for this, the statistics collection should be set up in a way that allows tracking the activity of a particular user).

It’s impossible to test a product without understanding your users, their values, and problems. And here, again, the knowledge of the practical use of statistics collection systems (both user and marketing) is relevant. Another important skill that will come in handy is the ability to turn the analyzed data into suggestions for product improvement.

Conclusion

A good product QA is not a simple performer or someone who defends every bug fix, writes tons of test documentation, can’t work without properly described requirements, and quits because of a “poor-quality” product. This is someone who balances the user and business, adapts the depth of testing and approach to their current goals and work conditions, and shows empathy and willingness to develop the product together with the Product Manager and other team members. Of course, being well-versed in various testing types and having a solid knowledge of the technical base and tools are important. But these skills won’t be so significant without certain personal competencies. However, that’s a different story… 🙂

More recommendations from AIBY experts in the next articles. Follow us for more!