What Is Pairwise Testing And How To Use It For Cost-Effective QA

Published in

--

Pairwise testing is a method of software testing that helps in reducing the number of tests while keeping the testing quality at a high level. We use this approach in development of 1C:Enterprise-based products, and today will show how this method could be used in the real world.

What is pairwise testing

Before exploring real-life applications of the pairwise testing let’s describe in more detail how this technique works.

Pairwise testing is a black box testing method. It is essential to understand that most common bugs that can be found in software are usually triggered by either a single input parameter or an interaction between pairs of parameters. Finding bugs involving interactions between three or more parameters is hard and costly.

Pairwise testing involves testing all possible discrete combinations of system’s each pair of input parameters instead of searching for all combinations of all parameters. By using carefully chosen test vectors and parallelizing tests, this method is capable of significant resource saving and speeding up the whole process.

As a result, the number of test-cases used is lowered while the overall functional coverage does not decrease significantly.

“Usual” approach to testing

To better understand all benefits and details of this technique it is a good idea to see how this works for QA of a real project. For example, we need to test possible ways of how document called Loan contract can work and function.

For the beginning, we need to define testing parameters and the volume of checks. Data and a loan period won’t be parameterized at this stage, for now, tests for values from the range of permissible values will suffice. We will search for boundary values and those within the range.

For example, the loan start date should not be older than the loan end date, etc. Here we have the following parameters:

Contract type: Lent or Borrowed

Currency: USD, Euro, CZK:

Fields Commission, Annual Interest, Principal have Yes or No values:

In the Repayment terms tab, the Payments field can have 5 values, while 360 Days in Year flag can be either set or unset:

Let’s combine all parameters in one table:

Simple parameters’ brute force gives us 2*3*5*2*2*2*2 = 480 overall tests number. Only for one document!

Pairwise testing for QA optimization

We can significantly reduce the number of checks using pairwise testing method. You can find tools for it here. We at 1Ci work with the PICT tool. To start using it, you’ll need to download it from GitHub, install the program, and create a txt file for input testing parameters.

It can be quickly done using Notepad. The number of rows in the document corresponds with the number of parameters. Their values go after the colon and divided by a comma.

Then create the Result.txt file and run the following command in command line:

As a result, the Result.txt file will be filled with shortened testing results.

As it can be clearly seen, we’ve managed to reduce the number of checks from 480 to 15. Such reduction yields significant time savings.

The pairwise testing method is highly effective for the cases with the limited number of parameters with multiple possible values.

More articles on software development and QA: