Test Design Techniques for Unit Testing

The Definition of Unit Testing

In his book The Way of the Web Tester, Jonathan Rasmusson writes:

Unit tests are small, method-level tests developers write to prove themselves that their software works.

I read three key points from this sentence:

1. Unit tests are method level tests that are aimed at testing one bit of functionality at a time. We can interpret these methods as “components”.
2. Unit tests reside closer to developers than to testers. However, a tester may assist a developer during test case design by applying specific techniques.
3. Unit tests prove the code rather than test it. Testing, in contrast, aims to establish in a series of experiments the basis for deciding whether the program works correctly and as expected by the customer . In other words, the unit tests are for verification but not validation.

The following definition arises from the three key points above:

Unit tests are component-level tests, engineered and implemented by developers, that test the components in isolation and aim to show that the code works according to software specifications.

What are components?

The term component comes from the “component architecture” approach, which states the following:

• Components are cohesive groups of code, in source or executable form, with well-defined interfaces and behaviors that provide strong encapsulation of their contents, and are, therefore, replaceable.
• Architectures based around components tend to reduce the effective size and complexity of the solution, and so are more robust and resilient.

As its name implies, the component architecture approach will therefore yield a “composition” of these components into progressively larger subsystems.

Figure 1, an example of component architecture.

Testing Techniques

The use & application of testing techniques to generate unit test will become more clear by means of relevant examples. Thus, for each testing technique I will go through an appropriate example. Since these sections will require a degree of intellectual effort, I would like to remind the reader that they may need to invest some time & understanding. Category Partition Method unfolds with an example on its own whereas Equivalence Value Partitioning & Boundary Value Analysis techniques are explained through contiguous examples.

Category Partition Method

The category paritition method is performed in three steps:

1. Analyze the specifications
2. Partition the category into choices
3. Determine the constraints to remove invalid combinations and to reduce the number of exceptional behaviors

Suppose its 1988 and you are working on a shell command that locates instances of a string argument in a target file. The following documentation has been passed to you:

Command:
find
Syntax:
find <pattern> <file>
Function:
The find command is used to locate one or more instances of
a given pattern in a text file. All lines in the file that contain
the pattern are written to standard output. A line containing
the pattern is written only once, regardless of the number of
times the pattern occurs in it.
The pattern is any sequence of characters whose length does
not exceed the maximum length of a line in the file. To
include a blank in the pattern, the entire pattern must be
enclosed in quotes (“). To include a quotation mark in the
pattern, two quotes in a row (**) must be used.
Examples:
find john my file
displays lines in the file myfile which contain john
find “john smith” myfile.
displays lines in the file myfile which contain john smith
find “john”” smith” myfile.
displays lines in the file myfile which contain john” smith

First identify the parameters by reading the specification:

• pattern size
• quoting
• embedded blanks
• embedded quotes
• and file name

Second partition the category into choices:

• pattern size: empty, single character, many character, longer than any line in the file
• quoting: pattern is quoted, pattern is not quoted, pattern is improperly quoted
• embedded blanks: no embedded blank, one embedded blank, several embedded blanks
• embedded quotes: no embedded quotes, one embedded quote, several embedded quotes
• and file name: good file name, no file with this name, omitted

A Cartesian product of these category options will generate all possible test cases.

Third eliminate supefluous cases by determining constraints:

• Treat “pattern is improperly quoted” category as exceptional, and test it once. Since if the text pattern is improperly quoted, the result will not change depending on the other categories.
• Treat pattern size “empty” as exceptional, and thus, test it just once. Since if the text pattern is improperly quoted, the result will not change depending on the other categories.

However, be careful ! As not all constraints are a friendly; one constraint that should not be added is:

• Constrain the “good file name” option to happen only if “pattern is not quoted” also happens. This is not a good constraint because a test case based on the category options a good file name and a quoted pattern, for instance, is a non-trivial test case.

Equivalence Value Partitioning

Equivalence Value Partitioning divides the input data into groups, where each group causes the program to return a corresponding output.

Suppose we are building chocolate packages:

A package should store a different number of kilos. There are small bars, 1 kilo each, and big bars, 5 kilos each. We should calculate the number of small bars to use, assuming we always use big bars before small bars. Return -1 if it can’t be done.

An implementation of this program is the following:

public class ChocolateBags {

    public int calculate(int small, int big, int total) {
        int maxBigBoxes = total / 5;
        int bigBoxesWeCanUse = maxBigBoxes < big ? maxBigBoxes : big;
        total -= (bigBoxesWeCanUse * 5);

        if(small <= total)
            return -1;
        return total;

    }

}

The equivalence classes for this function would be:

1. Only big bars. If the total package amount is 15 kg and there are 3 big bars and 2 small bars. The big bars will be enough to fill the package.
2. Only small bars. If the total package amount is 2 kg and there are 3 big bars and 2 small bars. Only small bars can be used to fill the package.
3. Big and small bars. If the total package amount is 17 kg and there are 3 big bars and 5 small bars. Both big and small bars need to be used to fill the package.
4. Total is greater than big and small bars. If the total package amount is 18 kg and there are 3 big bars and 2 small bars. Big and small bars together will not be enough to fill the package and the program will return -1.

A test is to be implemented for each of these equivalence classes. JUnit5 makes it possible to implement these test cases.

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

public class ChocolateBagsBeforeBoundaryTest {
    @Test
    public void totalIsBiggerThanAmountOfBars() {
        ChocolateBags bags = new ChocolateBags();
        int result = bags.calculate(2, 3, 18);
        Assertions.assertEquals(-1, result);
    }

    @Test
    public void onlyBigBars() {
        int result = new ChocolateBags().calculate(2, 3, 15);
        Assertions.assertEquals(0, result);
    }

    @Test
    public  void bigAndSmallBars() {
        int result = new ChocolateBags().calculate(5,3,17);
        Assertions.assertEquals(2,result);
    }

    @Test
    public void onlySmallBars() {
        int result = new ChocolateBags().calculate(2,3,2);
        Assertions.assertEquals(3,result);
    }

}

Boundary Value Analysis

However testing against these equivalent partitions are not enough. Consider the partition again with different slightly input:

• Big and small bars. If the total package amount is 17 kg and there are 3 big bars and 2 small bars. Both big and small bars need to be used to fill the package.

Here the expected value is 0 but the test will return -1. Thus there is a bug in the program.

So, what flips the result here? Closer inspection reveals that there is a boundary before and after which the result flips:

• If the total package amount is 17 kg and there are 3 big bars and 4 small bars. The result is 0.
• If the total package amount is 17 kg and there are 3 big bars and 3 small bars. The result is 0.
• If the total package amount is 17 kg and there are 3 big bars and 2 small bars. The result is -1.
• If the total package amount is 17 kg and there are 3 big bars and 1 small bars. The result is -1.

The following code block is responsible for setting this boundary:

if(small <= total)
            return -1;

The boundary presents itself once the small bar amount is equal to the remaining packageable amount. The program at that point starts to return -1 and keeps returning -1 thereafter as long as it is less than the remaining packageable amount.

In fact, the bug is due to equality part of the operator “less than or equal to” that sets the boundary. If the quantity of small bars equals the remaining total, then the program should return 0 not -1. In order to test against the boundary values, JUnit offers parametrized tests to check the program against multiple inputs:

//Name the parameters by positional arguments
    @ParameterizedTest(name = "small={0}, big={1}, total={2}, result={3}")
//Indicate parameters as comma separated strings
    @CsvSource({
        "0,3,17,-1", "1,3,17,-1", "2,3,17,2", "3,3,17,2",
        "0,3,12,-1", "1,3,12,-1", "2,3,12,2", "3,3,12,2"})
    public void bigAndSmallBars(int small, int big, int total, int expectedResult) {
        int result = new ChocolateBags().calculate(small, big, total);
        Assertions.assertEquals(expectedResult, result);
    }

This article here explores the boundary value analysis on its own and demonstrates the use of domain matrix in determining test cases.

The CORRECT Way

Boundary value analysis can be extended to address more conditions using the acronym CORRECT :

1. Conformance . Does the value conform to an expected format?
2. Ordering . Is the set of values ordered or unordered as appropriate?
3. Range . Is the value within reasonable minimum and maximum values?
4. Reference . Does the code reference anything external that isn’t under direct control of the code itself?
5. Existence . Does the value exist (e.g., is non-null, nonzero, present in a set, etc.)?
6. Cardinality . Are there exactly enough values? See the fencepost error under off-by-one errors in Wikipedia.
7. Time (absolute and relative) . Is everything happening in order? At the right time? In time? For instance how do you manage timeouts?

Sources

Disclaimer: the codeblocks used above are not mine. I took them from testing course from DelftX mentioned below. However, I did put in the effort to understand and appropriate the lessons therein. That’s why I feel that it’s ok to share them with you.

What are components?

my thanks to Herbert Barrientos

Category Partitioning:

https://www.researchgate.net/publication/220422305_The_Category-Partition_Method_for_Specifying_and_Generating_Functional_Tests

Equivalence & Boundary Value Analysis

Automated Software Testing: Unit Testing, Coverage Criteria and Design for Testability: https://learning.edx.org/course/course-v1:DelftX+ST1x+3T2022/home

The CORRECT Way

Pragmatic Unit Testing in Java 8 with JUnit by Jeff Langr, with Andy Hunt and Dave Thomas