Effective QA Best Practices

Published in

Effective Developers

6 min readApr 5, 2019

Working at Effective I’ve participated in more than 30 different projects. All of them were completely different: web and mobile, large and small, complicated and simple. We were building projects from scratch, adding new features and maintaining existing projects.

There were a lot of tricky cases in quality assurance process, testing, management and development and now it feels like our team passed Per Aspera ad Astra.

Currently, it feels like nothing special was done, but comparing Effective now and then I can say we made a tremendous step forward. Analyzing mistakes were made, team wishes and suggestions, I’ve prepared the following best-practices list for quality assurance. The purpose of this list is to help young teams understand what can be useful for them without spending much time on research. For some of experienced, this may look as bicycle re-invention :) Here we go!

Understand business goals and make Acceptance Criteria clear

This is the cornerstone of a project and should be clarified before the development starts. A project team has to understand what should be done, who is the target audience/feature consumer, and how to understand there is nothing left to do.

Understanding business goals will help you to answer questions like ‘What do we need to do and who will use it?’. Acceptance criteria will help you to understand does a feature meet business objectives or not. From my experience, I can say that business goals and acceptance criteria should be confirmed by a client or client’s representative. Additionally, based on clear and transparent business goals and acceptance criteria, skilled outsourcing software development companies can upgrade a business idea or suggest a better solution.

CI/CD — Continuous Integration/Continuous Deployment

Sounds familiar… Isn’t it? One of the trending things in the current software development the main advantage of which from the QA team perspective is that we can easily get a build with the latest features or fixes.

During the development cycle, our development team can have a lot of git branches related to different features but only one branch contains the most updated code. CI tool checks this branch and when the code is changed, it creates a new build and shares it to specified distribution service.

We’ve tried TeamCity and Jenkins. Both of these are great tools. TeamCity has more beautiful UI, but Jenkins is absolutely free, so we’ve chosen Jenkins.

Application distribution services

In general, it looks like nothing special but under the hood, continuous integration with tuned application deployment services gives you the easiest and fastest way to get the latest build on any testing device or environment you want. It’s ok to upload build via USB to one device. But what if you need to check the build using 10 different devices? That’s the point.

For mobile projects, we’ve tried a few different services like HockeyApp, Beta by fabric (ex-Crashlytics), Test Flight by Apple, Play Console by Google. Of course, there are more services but these have been chosen as the most popular. Now I’m voting for Test Flight and Play Console because of these services are flexible, support internal and external testers features and official services from Apple and Google and from testers email only is required. The only restriction here is that you need Apple and Google developer account which costs 25$ for Google (one-time payment) and 99$ for Apple (annually).

Other services like HockeyApp or Beta have some difficulties with adding of new testers to the project, especially on iOS. Due to Apple security care, from testers, it’s required to provide UDID of their devices to the developer and the developer has to add these UDIDs to project. Since we share dev builds with our clients (who usually have a lot of different devices and changing them regularly) all of us got tired of this UDID collecting activities. That’s why we have chosen Test Flight and Play Console.

For web projects, everything is a bit simpler because we use a special testing environment which is updated by the CI tool when the development branch is changed.

Testing Documentation

Through the years our QA team figured out four most valuable documents that can be shared with clients or stakeholders:

Supported platforms
Test Plan
Test Cases/Checklists
Release Notes

Supported platforms document should be prepared as early as possible when a project begins and signed with the client. It should contain information about supported hardware and software configurations, known limitations and restrictions. It’s also based on the target audience devices because devices markets are different in different countries.

Signing this with our clients, we guarantee that the first version of a product is working perfectly on listed configurations, besides we let our clients know that product can work on the other configurations but some issues may appear. And if the product and target audience will grow, we’ll be able to analyze and implement additional platforms support. In the future, this document will allow us to focus on specified platforms in development and bugfix.

Test plan is also should be prepared at the beginning and shared with the customer. This document should contain all types of testing that will be used during product development with a described purpose for each type. In the test plan, the QA team should decide what do they use for testing, test cases or checklists. Usually, it depends on project duration and functional complexity. Supported platforms should be linked to the test plan as well. And finally, the test plan should contain information about planned testing activities by dates following the project development and release timeline.

Test cases/checklists is a required thing for every project. Of course, it takes some time to prepare these deliverables and it can take additional time for support of this documentation, but it gives you some kind of tree trunk and using this trunk you can easily imagine and create new testing scenarios just by adding branches to your trunk. Later you can share prepared test cases with UAT team on the client’s side or with beta testers or even show test cases to the project development team. Dev team can use test cases as a part of requirements and it really can help them to avoid some issues.

At Effective we’ve tried a lot of TMS (Test Management System) and chose TestRail as one of the most popular, customizable, fast and convenient tools for test cases management and test management. Using TestRail allows us to easily keep test cases and checklists up to date. For us, this tool suites great, but there are still a lot of alternatives. The main tip here is to use the proper TMS and don’t use Google Docs and Spreadsheets for test cases and test logs :)

Release Notes is the document prepared by our QA team for clients and containing actual information about the project. What features were completed in the particular sprint, what is still in progress, what are known issues, where and how the demo build can be downloaded. We always prepare release notes by sprint and by release. It gives our clients additional transparency about development progress.

Exploratory Testing

The final thing that should be never forgotten is Exploratory Testing. The main purpose of this testing is to understand your product better and take a look at it from a user’s perspective. Combining exploratory and scripted testing (by scripted testing I mean test cases or checklists usage), combining tester’s and user’s perception of the product and keeping in mind business goals you can make the product you work on as perfect as possible.

As a part of exploratory testing, we also use swarm testing approach. Using Test Flight and Play Console we invite external testers who are usually Effective employees who are out of the project and can act as beta testers. This allows us to get a product overview from a user’s perspective and pay attention to usability.