On testing in DDD

Classes designed according to the Domain Driven Development principles may create some challenges concerning testing (e.g. https://stackoverflow.com/questions/63424912/tdd-ddd-model-abstractions/). When instantiating such classes you need to be especially careful as typically there are invariants, that, when not fulfilled, may result in tests failures. The classes are also often rich, they contain numerous fields (this can be a challenge not only in DDD). Arranging them in tests can easily result in tight coupling between tests and production code. That means each update in the production class leads to a change in the tests that use the class. We can work this out by creating reusable factories that respect the invariants and set all obligatory fields, but that does not remove the coupling. It only gathers it in a single place and makes it manageable. I faced the challenge a couple of years ago when working on an eCom product. The tests maintenance costs were so high during development that my team found time to stop for a moment and think about how we can improve the matters. And we found a solution. We created a tool for arranging random data for tests (https://github.com/ocadotechnology/test-arranger) and an approach inspired by the fixture patterns to organize the test code. It worked well for us then and since that moment I use it practically in every project I’m working on. So let me introduce you to a solution that saved me hours of boring tests maintenance.

Introducing the test-arranger

Let’s check how it works. The system under test is an eCom, and as every eCom it has products. For a start, they will be represented by the following class:

@Getter
@AllArgsConstructor
public class Product {
    @NonNull
    private String id;
    @NonNull
    private String name;
    @NonNull
    private String brand;
    @NonNull
    private BigDecimal price;
    @NonNull
    private TemperatureRegime temperatureRegime;
    @NonNull
    private BackOfPack backOfPack;
    private List<Image> images;
...
    public String shortName() {
        ...
    }
...
}

The Product contains a lot of fields, most of them guarded by invariants. Yet, there is nothing exceptionally fancy, only ordinary “nonnull”. Additionally, there is the shortName() method that deserves a decent unit test. A naive solution could be as follows:

@Test
void shortName() {
    //arrange
    var name = "name";
    var product = new Product("id", name, "brand", BigDecimal.TEN, TemperatureRegime.Chill, new BackOfPack(), null);

    //act
    var actual = product.shortName();

    //assert
    assertThat(name).contains(actual);
}

The solution is not perfect. The test requires only the name, but I gave all parameters which created a tight coupling between the test and the Product class. Tighter than it needs to be. The test uses only the name, so all remaining parameters can be considered accidental complexity that additionally blurs the intention which is assuring that the short name is correctly constructed from the name. Let’s check how it can be done with the test-arranger:

@Test
void shortNameArranger() {
    //arrange
    var product = some(Product.class);

    //act
    var actual = product.shortName();

    //assert
    assertThat(product.getName()).contains(actual);
}

We have there two important achievements. Loosening the coupling with production code: the signature of the production constructor is not relevant anymore and the test does not need to know anything about other Product parameters than the name. And the second thing is that the name parameter is not particularly relevant too — we do not set it explicitly. The object of the test is to assure that the shortName() method converts the name in a particular way. The exact value of the name is not relevant. Note that it is a manifestation of the rule about avoiding duplication in tests recommended by TDD (Kent Beck: Test Driven Development: By Example).

Random data in tests can reduce the coupling with the production code and hence make the test intention more obvious.

On the other hand, I simplified the task by giving out control over the name. What if our assertions cannot be narrowed to a simple predicate? What if there is some fancy natural language processing involved and we need to give a certain input value and expect a certain output to test it properly? The answers are not straightforward as the Product is immutable and hence once created cannot be altered (if the class under test is mutable, this is a no-brainer, thus despite it is not something unusual, I will not explore this scenario further). I’m used to solving this challenge with Lombok’s builder with the toBuilder parameter:

@Getter
@AllArgsConstructor
@Builder(toBuilder = true)
public class Product {
...

It is questionable whether we should extend production classes for the sake of tests. However, in this case, it is worth considering, as that is a change that can make the production code better. You can do much more with immutable classes with something similar to copy-constructor (e.g. in Kotlin’s data classes such feature was included by design) and that is exactly what the @Builder(toBuilder = true) delivers. With this annotation, we can call toBuilder() method on the Product instance to get a builder initialized with the original Product values. Then we can change using the builder the parameter we are interested in and create an altered Product copy. Let’s use that in our test:

@Test
void shortNameArranger2() {
    //arrange
    var product = some(Product.class).toBuilder()
            .name("Great Value Frozen Pineapple Chunks 500g")
            .build();

    //act
    var actual = product.shortName();

    //assert
    assertThat(actual).isEqualTo("Frozen Pineapple");
}

So I use the test-arranger to create a random instance of Product (no coupling with the constructor signature or class parameters) and then I use toBuilder() method to set in the name the certain value (“Great Value Frozen Pineapple Chunks 500g”) I need for my test case. As a consequence, the coupling with the production code tighten as little as it was possible. The test case depends on the name, but it cannot be avoided as it is part of the object of the test. Additionally, there is no boilerplate code in the test. Practically, everything that is in the test is necessary. As a consequence, it is not challenging to understand the test intention just by reading the code of the test, i.e. that shortName() method should produce “Frozen Pineapple” when its input equals “Great Value Frozen Pineapple Chunks 500g”.

Lombok’s @Builder(toBuilder = true) can be very handy when arranging instances of immutable classes.

Kotlin approach

In Kotlin, there is an interesting alternative — the data classes. They come with the very handy copy() method that makes it possible to do a similar thing as with the Lombok’s builder. So if the Product was a Kotlin data class, the “arrange” part of the test could be as follows:

val product = some<Product>()
           .copy(name = "Great Value Frozen Pineapple Chunks 500g")

Unfortunately, Java’s counterpart (the record) is not so powerful, and we still need to use the builder.

When data class is not an option, but the class under test does not have to be immutable, another concise Kotlin syntax can be used:

val product = some<Product> {
    name = "Great Value Frozen Pineapple Chunks 500g"
}

Invariants

We have just seen how test-arranger can be handy in simple cases, but in DDD we often face more challenging situations. The random data may not be in line with domain invariants and as a consequence, we may fail in creating instances required for the tests or create invalid objects that will be rejected by validation rules. Let’s assume that the Product needs to be extended with a list of Barcodes and a feature for checking if a given Barcode is associated with the Product:

public class Product {
...
    private List<Barcode> barcodes;
...
    public boolean hasBarcode(Barcode barcode) {
...
}
@Getter
@AllArgsConstructor
@Builder(toBuilder = true)
public class Barcode {
    @NonNull
    private String code;
    @NonNull
    private BarcodeType type;
}

public enum BarcodeType {
    Code39, Code93, Code128, Ean8, Ean13, Jan, UpcA, UpcE
}

The Barcodes are of certain types that impose rules on the code content (code field in the listing above), e.g. the expected number of characters. If the code field is initialized with a random sequence of characters, the logic depending on Barcodes will not work as it should. And its tests will manifest the garbage in — garbage out pattern. Not what a respected tester wants.

The hasBarcode() method is not as simple as it seems at the first look. Barcode is to some extent product identifier. When paying for the shopping in a grocery, the cashier scans the barcodes to let the system find out which products were bought. However, some barcodes contain a dynamic part that encodes for instance the expiry date. The Product class represents a certain type of inventory, so it will not be suitable to store Barcodes with each possible expiry date. Instead, a so-called canonical version is used — it has no real expiry date, but it is possible to compare it against the real barcode to determine if it represents the same kind of product. So in the Product.barcodes, there are canonical versions of the barcodes. However, the hasBarcode() method should return true not only when there is an exact match, but also when in the Product, there is a canonical version corresponding with the real barcode (containing an expiry date) used as the method parameter.

That constitutes a feature that depends on the validity of the Barcode — a Barcode cannot be compared against its canonical version when it is just a random sequence of characters. Therefore, to have valid data in the tests, a recipe for creating correct Barcodes must be delivered to the test-arranger. It can be done by extending the CustomArranger class:

class BarcodeArranger extends CustomArranger<Barcode> {

    @Override
    protected Barcode instance() {
        Barcode barcode = super.instance();
        return barcode.toBuilder()
                .code(generateCanonicalCode(barcode.getType()))
                .build();
    }

    private String generateCanonicalCode(BarcodeType type) {
        ...
    }
}

The body of the generateCodeOfType() method was omitted for simplicity. Test-arranger is looking for the custom arranger using reflection. To not scan the whole classpath, which can cost a lot of time, there is the arranger.root property that should point to a package covering all custom arranger, e.g.:

arranger.root=com.your_company_name

The BarcodeArranger will be picked up by the test-arranger whenever a new instance of Barcode is created. The test-arranger will call the instance() method and use its result. When the instance() method isn’t overridden, a completely random instance is created. Test-arranger uses custom arranger not only when direct instantiation is requested. That means, it will be used in both:

var product = some(Product.class);
var barcode = some(Barcode.class);

When a class cannot be entirely random, provide a custom arranger (should extend the CustomArranger) that instantiates the class less randomly, without breaking its invariants.

Let’s go back to the test case. With the custom arranger for Barcode, it can be as follows:

@Test
void hasNotBarcode() {
    //arrange
    Barcode barcode = some(Barcode.class);
    Product product = some(Product.class);
    
    //act
    boolean actual = product.hasBarcode(barcode);
    
    //assert
    assertThat(actual).isFalse();
}

It will work like a charm, as the test-arranger already knows how to create valid Barcodes however, still uses random data. First, a valid but random Barcode is created. In the next line, a Product is instantiated. The Product has a list of Barcodes, each of them will be valid. But since random data is used, we will not find there the Barcode that was created in the previous line.

The positive scenario can be tested similarly:

@Test
void hasBarcode() {
    //arrange
    Barcode barcode = some(Barcode.class);
    Product product = some(Product.class).toBuilder()
            .barcodes(List.of(some(Barcode.class), barcode))
            .build();

    //act
    boolean actual = product.hasBarcode(barcode);

    //assert
    assertThat(actual).isTrue();
}

It definitely should be tested that the hasBarcode() method works correctly with Products containing the canonical Barcodes. Arranging data for such a test case is a bit more challenging. We need a Product associated with a canonical Barcode and a real Barcode that is in line with the canonical one. To get there it would be convenient to have a method that can generate a canonical one and a method that generates a real one, with a random code, but according to the given canonical barcode. Both methods are about arranging Barcodes, so why not put them into BarcodeArranger:

class BarcodeArranger extends CustomArranger<Barcode> {

    @Override
    protected Barcode instance() {
        Barcode barcode = super.instance();
        return barcode.toBuilder()
                .code(generateCodeOfType(barcode.getType()))
                .build();
    }

    public static Barcode canonical() {
        ...
    }
    
    public static Barcode inLineWith(Barcode canonical) {
        ...
    }
    
    private String generateCodeOfType(BarcodeType type) {
        ...
    }
}

Please note that the new methods are public static. They do not share any state with the BarcodeArranger class, nonetheless, I decided to put them into arranger. Such an approach makes it obvious where to look for factory methods when creating objects for tests — all of them are in the NameOfClassArranger. It is a small thing, but it helps in getting rid of duplication in tests.

All methods designed for instantiating a certain class (for the sake of the tests) should be moved to the class arranger. Moreover, the methods should be reused (whenever suitable) to avoid duplication in the test data arrangement.

With the convenient BarcodeArranger, I can finally create the test case:

@Test
void hasCanonicalBarcode() {
    //arrange
    Barcode canonical = BarcodeArranger.canonical();
    Barcode barcode = BarcodeArranger.inLineWith(canonical);
    Product product = some(Product.class).toBuilder()
            .barcodes(List.of(some(Barcode.class), canonical))
            .build();

    //act
    boolean actual = product.hasBarcode(barcode);

    //assert
    assertThat(actual).isTrue();
}

Kotlin approach

In Kotlin, the arranger classes can be usually simpler. Typically, different test cases require setting different subsets of the class fields. In Java, we cannot do much besides writing corresponding with the needs methods that increase arranger complexity. Fortunately, in Kotlin, there are named and default arguments. So we can come up with the following BarcodeArranger (it does not address the canonical barcode challenge we explored before):

class BarcodeArranger {
    companion object {
        fun with(
            code: String = some(),
            type: BarcodeType = some()
        ): Barcode =
            Barcode(code, type)
    }
}

The above BarcodeArranger creates completely random Barcodes. In real life, there would be some fancy logic (e.g. converting codes to canonical ones according to the selected type) — the purpose of creating an arranger in the first place. But my only goal here is to show how Kotlin is handy in arrangers, and hence no additional details are given. And the power of Kotlin comes from the fact that with the one simple methods from the previous listing, all those calls are possible:

val b1 = BarcodeArranger.with(code = "my_fancy_code")
val b2 = BarcodeArranger.with(type = BarcodeType.Ean13)
val b3 = BarcodeArranger.with(code = "my_fancy_code", type = BarcodeType.Ean13)

We do not face here the unwanted explosion of methods covering different combinations of necessary arguments. We can (usually) have just one, where each not given explicitly argument is assigned with a random value (the some() method call).

Beyond single aggregate

Till now all the examples were focused on arranging a very narrowed number of classes that can be generalized to nothing more than a single aggregate. Unfortunately, software systems usually are not so simple. Often there is more than one aggregate, and there are test cases depending on interactions between different aggregates. That isn’t something we can well address with arrangers — an arranger, by the definition, is associated with a certain class. When there are two aggregates to arrange which of the two arrangers should contain the code? Fortunately, there is a well-known tests design pattern — the fixture. It was described in many variants by Gerard Meszaros in xUnit Test Patterns: Refactoring Test Code (worth reading to find more details about the pattern).

Let’s imagine that there are shopping aisles in our shop and each product is assigned to one or more aisles. On the other hand, the Aisles have some attributes, like description, and create a tree-like structure. As a consequence, they were designed as a different aggregate than the Product class we already tested before. So the Product is extended with the following field:

private List<AisleId> aisles;

And a new class to represent the aisles:

@Getter
@AllArgsConstructor
@Builder(toBuilder = true)
public class Aisle {
    private AisleId id;
    private String name;
    private String description;
    private List<Aisle> subaisles;
}

When the user selects a certain Aisle, the system should render all the Products assigned to the selected Aisle and all its sub-aisles. The algorithm should go deeper and deeper into the sub-aisles till it reaches the leaves (Aisles without sub-aisles). It sounds like a feature where regression may happen and hence should be guarded by some tests (the method signature suggests that there is a design issue that may result in poor performance, but let’s ignore that and focus on the testing):

public List<Product> filterProductsByAisle(
      List<Product> products, 
      Aisle aisle) {
    ...
}

We can easily get away with random data when testing that the method filters out Products not assigned to the given Aisle:

@Test
void filterProductsNotAssignedToAisle() {
    //arrange
    var aisle = some(Aisle.class);
    var products = someObjects(Product.class, 10)
            .collect(Collectors.toList());

    //act
    var actual = sut.filterProductsByAisle(products, aisle);

    //assert
    assertThat(actual).isEmpty();
}

In the test, a random Aisle is created, and then ten random Products. The first has nothing to do with the second, so we can assert that the result of filtering is an empty collection. However, to test the positive scenario we need Products containing the ids of the Aisles we submit for filtering. That’s finally the moment when the fixture comes into play:

public class ProductsAndAislesFixture {

    public final Aisle aisle;
    public final List<Product> products;

    public ProductsOnAislesFixture(int noOfProducts) {
        aisle = some(Aisle.class);
        Set<AisleId> aislesIds = aisle.allIds();
        products = someObjects(Product.class, noOfProducts)
                .map(p -> p.toBuilder()
                        .aisles(List.of(someFrom(aislesIds)))
                        .build())
                .collect(Collectors.toList());
    }
}

And the corresponding test:

@Test
void filterProductsAssignedToAisle() {
    //arrange
    var noOfProducts = somePositiveInt(20);
    var fixture = new ProductsOnAislesFixture(noOfProducts);

    //act
    var actual = sut.filterProductsByAisle(
            fixture.products, 
            fixture.aisle);

    //assert
    assertThat(actual).hasSize(noOfProducts);
}

When test data arrangement is covered by the fixture, the test becomes very straightforward, and its intention is quite easy to guess just by reading the code.

When a test requires complex data (more than one aggregate) store the data creation procedure in a fixture.

The fixture was created for only one use case, so it is a bit oversimplified. In an enterprise setup we should think about data immutability or about removing the data creation procedure from the constructor as with the forthcoming test cases we may need slight differences in the arrangement. When the data needs to be arranged in a different way, it is usually worth it to do it with a dedicated method whose name explains the peculiarities of the created data.

We can also think about the lifecycle of the fixture. When the data creation is very time-consuming, it can be justified to share the fixture between test cases. Consider creating data in an embedded database using Spring repositories… The spectrum of possible fixture variants is comprehensively covered by Meszaros in the book I mentioned a couple of passages earlier.

Conclusions and summary

DDD defines a number of building blocks representing different levels of size and complexity. I gave examples showing how to deal with them starting from primitives and ending with collections of aggregates which pretty much correspond with the bounded contexts. A concise summary of this approach is given in a diagram in the test-arranger readme:

source: https://github.com/ocadotechnology/test-arranger

That is an approach I have used for a couple of years and it serves me well. Nonetheless, it does not contain only advantages. Let’s note the observed pros and cons.

Test-arranger pros

Looser coupling with production code. The tests do not depend on complex constructors or implementation details (e.g. required fields) of the production classes. Under the hood, the test-arranger uses reflection to instantiate classes. Hence, it can automatically adapt to changes as long as they have nothing to do with the test goal, but that’s the level of coupling we need to accept.

Less code in tests. The test-arranger does the dirty job of delivering values for all required fields. While in the test remains only what is relevant for the test goal, and we end up with less code.

Clearer test intention. As it was already mentioned, with the test-arranger, we have in the tests only the relevant parts. It is much easier to discover the original intention without all the boilerplate code that vanished.

Additional possibilities for detecting issues. Random data is used to cover fields irrelevant to a given test case. However, the code under tests may be using the data incorrectly, and for a certain value, a variable that should be irrelevant results in the test failure. That is a good thing because it leads to detecting defects in unexpected places. Maybe it won’t happen on the first try, but that is still better than not detecting the defects at all. Consider the case with enumerations in the contract tests: https://stackoverflow.com/questions/43089653/should-i-test-all-enum-values-in-a-contract/66040704#66040704

Fewer duplications in the test code. Organizing the code for arranging test data into arrangers and fixtures makes it obvious where to look for a certain factory method. It does not prevent code duplication, but at least creates opportunities for reuse.

Lower test maintenance cost. I already mentioned looser coupling with production code, less test code, fewer duplication and clearer intentions. That all makes the need to adjust tests to changes in production code less frequent, and when it occurs, the tests should be easier to understand for the maintainer.

Test-arranger cons

More challenging debugging. The fields initialized with random data usually have fancy values, which are harder to memorize, and as a consequence, tracking the data flow may become difficult.

Blinking tests. If the expectations about what is relevant and what is irrelevant are wrong (the author misunderstood the code under test) and random data is used in a place where a certain value is needed, the test eventually will fail. It will happen when unacceptable data is drawn. This drawback is a counterpart of the Additional possibilities for detecting issues. When a mistake is in a test, the random data may sometimes compensate for it and result in success. When the mistake is in production code, random data may shed new light on it leading to a bugfix. From my personal experience, both situations are rare, and the additional possibility of blinking only increases my confidence in non-blinking test suites.

Slower tests. Generating random data is not as fast as assigning values to variables. It is still only a fraction of a second, but unfortunately, observably slower.

I hope that my ideas about organizing test data will help in solving challenges related to complex test data arrangements and reduce the number of tests that need to be adjusted in response to changed production code. If all the fixtures and arrangers look like too much fuss, try something simple. It isn’t all or nothing solution. If you use random data in a single test case, the benefit will be very small. But it will be there. So you can travel step by step at a pace that is convenient for you and your project.

Good luck in testing DDD.