Property-based Testing
Introduction
The purpose of this article is to present a brief introduction to property-based testing, what it is and how it’s different from traditional testing. For the examples of this blog, we will use ScalaTest and ScalaCheck.
Property-based Testing is a style of testing in which the strategy is to test properties of a piece of software, where a property refers to a behavior that code must obey. Instead of testing with concrete values as we would do in traditional testing, in property-based testing we let the test framework generate random values and check the expected results. One of such frameworks is ScalaCheck. This style of testing leads to tests that looks more like an executable specification, this is, a description of an expected behavior of the software that can be verified automatically.
Comparison with traditional testing
Let’s look at an example to better understand how property-based differs from traditional testing. Suppose we have a class Route with a priority:
And a class RoutesPlanner that order the routes based on the priority:
Using the traditional testing style, we could test the RoutesPlanner using something like the following code written with ScalaTest:
In the previous code, we test that if a first route has a higher priority than a second route then it should be the first element of the list returned by the RoutesPlanner.
If we want to test the case in which the second route has the highest priority, then we could add a second test:
In both cases, we provided specific values for the tests.
In contrast, using the property-based style we can test the specification that the route with the highest priority is always returned first, and we can replace the previous two cases with the following test written using ScalaCheck and ScalaTest:
In the previous example, we don’t provide concrete values for the attribute priority of the test. Instead, we use the ScalaCheck Gen Class to generate random values for the priorities with the method Gen.choose, and then with the function forAll we check that for all the generated values, the property is accomplished. When the test runs, ScalaCheck generates multiple random values and checks if for some value the test fails.
Generators
There are different types of generators, like generators for numbers, strings, lists, etc. We can also create our own generators and compose them to create more complex generators from simple ones.
For example, suppose we have a Payment class defined like:
Then we can create a Payment generator Gen[Payment] like this:
Here the generator Gen[Payment] is composed from the generators Gen.alphaNumStr, Gen.choose and Gen.oneOf.
Now suppose we have a Class called PricingCalculator, which given a Payment and a PricingConfiguration, should calculate additional values for the payment:
To test the method calculate we can use the generator Gen[Payment] together with another generator Gen[PricingConfiguration] for the pricing configuration:
In the following example, we will test the property that given a payment and a pricing configuration, a commission is calculated by the PricingCalculator:
As you can see, in one test we are testing the calculation of the commission for different payments values and different pricing configurations. We are also testing boundary conditions, in this case, that the commission is never outside the min and max commission limits.
Conclusion
With property-based testing, we can create tests as specifications, focusing on the behaviors that we want to verify, and letting the framework generate the values for the tests. This style of testing allow us to cover more code with fewer tests because one single test can check multiple values when it runs, and at the same time allow us to check more scenarios like boundary conditions with less code.
