Java: On The Benefits of Treating DTOs as Magic Cookies

Orren Chapman
Aug 31, 2018 · 10 min read

An examination of complex data handling and robust Java application development.

My View

There’s nothing magical about data transfer objects (DTOs); in their purest Java language form they contain a set of typed data fields with no responsibilities beyond aggregation. Treated as the simplest form of JavaBeans, DTO methods consist of getters and setters for their fields, with perhaps customized equals, hashCode and toString implementations.

Magic cookies in computer science parlance, are even simpler in purpose than DTOs: magic cookies are opaque blobs of data with no fields, no getters and no setters. While virtually meaningless in one context, they presumably are quite meaningful in another. As such, magic cookies are a kind of split-personality DTO: no meaning here, lots of meaning somewhere else.

Problem Statement

Within a distributed application architecture, DTOs carry the information between the application tiers. Because of the resource costliness of each individual communication between those tiers (e.g., latency, bandwidth), the resulting DTOs are often packed with lots of data representing complex information models. These models may be highly hierarchical in structure, a.k.a., “deep”: e.g., a set of things containing one or more lists of other things that contain even more things, on and on.

To address contextualized business inquiries (e.g., is this product transactable online?) may require traversal of many layers of these DTO hierarchies. And, given the nature of time based transactions, at each stage of those traversals, the next level in the hierarchy may be incomplete in whole or part. When working in Java these traversals if not carefully programmed can result in runtime exceptions; e.g., a NullPointerException.

Unit tests are a strategy to minimize application implementation risks; these tests require input data that will exercise the critical pathways in the application code and the broad range of possible inputs that may occur in production environments. Constructing and maintaining these complicated DTO hierarchy inputs very quickly becomes cumbersome and leads to complex test setups exceeding that of the application code itself; and, which may well obscure any assurance that the test results actually provide thorough application code coverage.

Object Alpha has a field beta that has a field gamma which has a String field containing a description. It's a simple DTO hierarchy and far more complex things exist in nature.

public class Alpha {
private Beta beta;
public Beta getBeta() {
return beta;
}
public void setBeta(Beta beta) {
this.beta = beta;
}
}
public class Beta {
private Gamma gamma;
public Gamma getGamma() {
return gamma;
}
public void setGamma(Gamma gamma) {
this.gamma = gamma;
}
}
public class Gamma {
private String description;
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
}

Then the “safe” or “bulletproof” Java code implementation to access the description field given the alpha input object would look something like:

if (alpha != null &&
alpha.getBeta() != null &&
alpha.getBeta().getGamma() != null &&
alpha.getBeta().getGamma().getDescription() != null) {
// ...
}

Some Common [Painful] Unit Test Solutions

Common strategies for unit testing applications accessing complex DTO hierarchies include mocking the test data hierarchy and assembling an inventory of canned test data inputs. Both these solutions succeed in the purpose but at considerable expense in terms of implementation and maintenance and both fail to provide the assurance of truly comprehensive coverage.

Mocking complex DTOs provides a versatile approach for verifying just about any application test scenario. Building out these mocked DTOs may require a great deal of test setup code in itself. And, as the complexity of the DTO hierarchy increases, the number of test scenarios likewise increases.

@Mock
private Alpha alpha;
@Mock
private Beta beta;
@Mock
private Gamma gamma;
@BeforeMethod
public void setUp() {
initMocks(this);
doReturn(beta).when(alpha).getBeta();
doReturn(gamma).when(beta).getGamma();
doReturn("description").when(gamma.getDescription());
}

That covers the happy path, but, one would have to account for the other scenarios: alpha is null, beta is null, gamma is null, or description is null.

Preserving DTOs in serialized forms (e.g., JSON) to be loaded as test inputs limits the amount of test setup code that mocking requires. This approach instead requires identifying and preserving the test inputs and then maintaining them overtime. When the data inputs are complicated these serialized versions may be quite large; and, as more scenarios are identified many copies may need to be maintained.

JSON found in alpha.json.

{
"beta": {
"gamma": {
"description": "..."
}
}
}

Once again, this covers the “happy path” scenario.

A Magic Cookie Approach

By treating DTOs as opaque things, i.e., magic cookies, solves both the complexity of accessing data and its evaluation. The following steps describe how.

Separating the logic surrounding accessing data from evaluating the results of that access, provides a means of comprehensively verifying these two different types of logic independently of one another. Access functionality then is encapsulated in a separate class in which the methods implement the different types of access on the corresponding DTO type.

An AlphaAccess class models the types of access on the Alpha "magic cookie". It is a singleton.

public class AlphaAccess {
private static final AlphaAccess INSTANCE = new AlphaAccess();
protected AlphaAccess() {
}
public static AlphaAccess getInstance() {
return INSTANCE;
}
public Beta getBeta(Alpha alpha) {
return alpha != null ? alpha.getBeta() : null;
}
}

An Access class method that returns a null value doesn’t mitigate the risk of programmer error when accessing data. Java 8 introduced the Optional class that can be used to eliminate null return values on Access methods.

The updated version of the getBeta method implemented using the Optional return value:

public Optional<Beta> getBeta(Alpha alpha) {
return alpha != null ?
Optional.ofNullable(alpha.getBeta()) :
Optional.empty();
}

Access methods are “go-fer” style methods that thanks to Java 8 Method References, can be greatly simplified using an abstract superclass:

An abstract superclass of a specific DTO Access class provides a common implementation to “get” a field value:

public abstract class AbstractAccess<T> {    protected <R> Optional<R> get(T o, Function<T, R> getter) {
return o != null ?
Optional.ofNullable(getter.apply(o)) :
Optional.empty();
}
}

And the AlphaAccess subclass is then implemented as:

public class AlphaAccess extends AbstractAccess<Alpha> {
...
public Optional<Beta> getBeta(Alpha alpha) {
return get(alpha, Alpha::getBeta);
}
}

And, given a separate unit test for the AbstractAccess class, the unit test for the AlphaAccess class is straightforward, too:

public class AlphaAccessTest {

private AlphaAccess alphaAccess;

@Mock
private Alpha alpha;
@Mock
private Beta beta;
@BeforeMethod
public void setUp() {
initMocks(this);

alphaAccess = new AlphaAccess();
}
@Test
public void testGetBeta() {
// given
doReturn(beta).when(alpha).getBeta();

// when
Optional<Beta> actual = alphaAccess.getBeta(alpha);

// then
assertEquals(actual, Optional.of(beta));
}
}

This provides 100% code coverage for the access logic contained within AlphaAccess.

Having created access methods on top-level fields, additional methods for fields accessed via those next level objects can be created at this top level, too, simplifying application access and verification code.

AlphaAccess has a new injected dependency, BetaAccess, to which it will delegate responsibilities for its data, and, BetaAccess, too, has an injected dependency, GammaAccess, to which it will delegate responsibility for access to its description field:

private static final AlphaAccess INSTANCE =
new AlphaAccess(BetaAccess.getInstance());
// injected dependency
private final BetaAccess betaAccess;
protected AlphaAccess(BetaAccess betaAccess) {
this.betaAccess = betaAccess;
}
...public Optional<String> getDescription(Alpha alpha) {
return getBeta(alpha).flatMap(betaAccess::getDescription);
}

In BetaAccess has a similar implementation:

public Optional<String> getDescription(Beta beta) {
return getGamma(beta).flatMap(gammaAccess::getDescription);
}

And, finally in GammaAccess we have:

public Optional<String> getDescription(Gamma gamma) {
return get(gamma, Gamma::getDescription);
}

And, now the unit test for AlphaAccess would include:

    @Test
public void testGetDescription() {
// given
doReturn(beta).when(alpha).getBeta();
doReturn(Optional.of("description")).when(betaAccess)
.getDescription(beta);

// when
Optional<String> actual = alphaAccess.getDescription(alpha);

// then
assertEquals(actual, Optional.of("description"));
}

Note, that betaAccess is a mocked object of type BetaAccess, injected at AlphaAccess construction and stubbed here to return an Optional object of the description field “description” String value. This stubbing is not unit test cheating in that the behavior is outside the scope of the AlphaAccess class itself whose only responsibility is to retrieve the beta object and delegate to its Access class, BetaAccess, the responsibility of obtaining the Optional object containing the description field's value.

Now, our application code can safely implement its function when a value is present and without incorporating all the complexities associated to obtaining that value.

The Optional object enables NullPointerException-free access code.

public class App {
// injected dependency
private final AlphaAccess alphaAccess;
private final Alpha alpha; public App(Alpha alpha) {
this(AlphaAccess.getInstance(), alpha);
}
protected App(AlphaAccess alphaAccess, Alpha alpha) {
this.alphaAccess = alphaAccess;
this.alpha = alpha;
}
public void doSomething() {
alphaAccess.getDescription(alpha).ifPresent(description -> {
// ...
});
}
}

And the application unit test then is simplified because only the alpha “magic cookie” and its corresponding access class need be mocked and stubbed.

public class AppTest {
private App app;
@Mock
private AlphaAccess alphaAccess;
@Mock
private Alpha alpha;
@BeforeMethod
public void setUp() {
initMocks(this);
app = new App(alphaAccess, alpha);
}
@Test
public void testDoSomething() {
// given
doReturn(Optional.of("description")).when(alphaAccess)
.getDescription(alpha);

// when
app.doSomething();

// then ...
}
}

Achieving 100% application code coverage is much more readily achievable now that the AlphaAccess class encapsulates the underlying details of how the description field value was retrieved; the unit test can focus on the evaluation of the results instead of the complexity of building out the complex DTO inputs.

It maybe useful to universally treat empty Collection subclasses and Map implementations and blank String objects as missing (Optional.empty()). This can be accomplished easily in the Access superclass.

The AbstractAccess class is updated to filter out empty Collection and Map instances and blank String instances:

public abstract class AbstractAccess<T> {    protected <R> Optional<R> get(T o, Function<T, R> getter) {
return o != null ?
Optional.ofNullable(getter.apply(o))
.filter(r -> !(r instanceof String) ||
StringUtils.isNotBlank((String) r))
.filter(r -> !(r instanceof Collection) ||
CollectionUtils
.isNotEmpty((Collection<?>) r))
.filter(r -> !(r instanceof Map) ||
MapUtils.isNotEmpty((Map<?, ?>) r)) :
Optional.empty();
}
}

Finally, it is often useful to support different Collection-oriented filters. Testing whether all members of a Collection match a particular criteria (Predicate) or any do or none do, finding one that matches or finding the first one that matches

Once again the AbstractAccess superclass is updated to provide support for different type of Stream-oriented access types: allMatch, anyMatch, findAny, findFirst, noneMatch.

public abstract class AbstractAccess<T> {    protected boolean allMatch(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) &&
collection.stream().allMatch(predicate);
}
protected boolean anyMatch(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) &&
collection.stream().anyMatch(predicate);
}
protected Optional<T> findAny(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) ?
collection.stream().filter(predicate).findAny() :
Optional.empty();
}
protected Optional<T> findFirst(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) ?
collection.stream().filter(predicate).findFirst() :
Optional.empty();
}
protected <R> Optional<R> get(T o, Function<T, R> getter) {
// ...
}
protected boolean noneMatch(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) &&
collection.stream().noneMatch(predicate);
}
}

By separating data access logic from data evaluation logic we are able to implement more reliable and maintainable software. The complexity of the data model no longer obscures the complexity of the application itself. And, verifying the two types of logic in isolation allows for a comprehensive examination of the important scenarios that need to be addressed in both domains, access and evaluation.

Next Steps: Java 10 Extensions

With Java 10, the Optional class has some important extensions which will be very useful in conjunction with the Access class strategy described here; specifically the ifPresentOrElse, or and stream methods will help data evaluation code address common scenarios.

Some application code where the absence of a particular value is as significant as its presence.

alphaAccess.getDescription(alpha).ifPresentOrElse(description -> {
// do something with the description ...
}, () -> {
// do something when there is no description ...
});

Some data scenarios have alternative resolution strategies which may be encapsulated within the Access classes themselves. In the case of the Gamma class with two fields, description and backupDescription.

public class Gamma {
private String description;
private String backupDescription;
// ...
}

If the rule is: when the description field is blank then the backupDescription field should be used; then, the GammaAccess class might encapsulate this rule by implementing:

    public Optional<String> getDescription(Gamma gamma) {
return get(gamma, Gamma::getDescription)
.or(() -> get(gamma, Gamma::getBackupDescription));
}

When dealing with collections, we often must deal with a Stream of Optionals, Java 10 helps simplify by dereferencing the non-empty instances:

Stream<Optional<Alpha>> alphaStream = ...;
List<String> descriptions =
alphaStream
.flatMap(Optional::stream)
.map(alphaAccess::getDescription)
.flatMap(Optional::stream)
.collect(Collectors.toList());

Appendix: AbstractAccess Unit Test

Here’s the source for the AbstractAccess super class which provides 100% code coverage for its helper methods so that the Access subclasses can focus for the most part on just their own "happy path" scenarios.

import java.util.List;
import java.util.Map;
import java.util.Optional;
import org.mockito.Mock;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import com.google.common.collect.ImmutableList;
import com.walmart.terrene.TerreneTestCase;
import static org.mockito.Mockito.doReturn;
import static org.mockito.MockitoAnnotations.initMocks;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertFalse;
import static org.testng.Assert.assertTrue;
public class AbstractAccessTest extends TerreneTestCase { protected interface TestObject {
List<Object> getList();
Map<Object, Object> getMap(); Object getObject(); String getString();
}
private static final String BLANK = "";
private static final String NOT_BLANK = "not-blank";

private AbstractAccess<TestObject> abstractAccess;

@Mock
private TestObject testObject;
@Mock
private Object object;
@Mock
private List<Object> objects;
@Mock
private Map<Object, Object> objectMap;
@BeforeMethod
public void setUp() {
initMocks(this);
abstractAccess = new AbstractAccess<TestObject>() {
};
}
@Test
public void testAllMatch() {
// then
assertTrue(abstractAccess.allMatch(
ImmutableList.of(testObject),
o -> true));
}
@Test
public void testAllMatchEmpty() {
// then
assertFalse(abstractAccess.allMatch(
ImmutableList.of(),
o -> true));
}
@Test
public void testAllMatchFalse() {
// then
assertFalse(abstractAccess.allMatch(
ImmutableList.of(testObject),
o -> false));
}
@Test
public void testAnyMatch() {
// then
assertTrue(abstractAccess.anyMatch(
ImmutableList.of(testObject),
o -> true));
}
@Test
public void testAnyMatchFalse() {
// then
assertFalse(abstractAccess.anyMatch(
ImmutableList.of(testObject),
o -> false));
}
@Test
public void testAnyMatchNull() {
// then
assertFalse(abstractAccess.anyMatch(
null,
o -> true));
}
@Test
public void testFindAny() {
// then
assertEquals(abstractAccess.findAny(
ImmutableList.of(testObject),
o -> true),
Optional.of(testObject));
}
@Test
public void testFindAnyEmpty() {
// then
assertEquals(abstractAccess.findAny(
null,
o -> true),
Optional.empty());
}
@Test
public void testFindFirst() {
// then
assertEquals(abstractAccess.findFirst(
ImmutableList.of(testObject),
o -> true),
Optional.of(testObject));
}
@Test
public void testFindFirstEmpty() {
// then
assertEquals(abstractAccess.findFirst(
null,
o -> true),
Optional.empty());
}
@Test
public void testGetBlank() {
// given
doReturn(BLANK).when(testObject).getString();
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getString),
Optional.empty());
}
@Test
public void testGetEmptyMap() {
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getMap),
Optional.empty());
}
@Test
public void testGetNotBlank() {
// given
doReturn(NOT_BLANK).when(testObject).getString();
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getString),
Optional.of(NOT_BLANK));
}
@Test
public void testGetNotEmptyList() {
// given
doReturn(objects).when(testObject).getList();
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getList),
Optional.of(objects));
}
@Test
public void testGetNotEmptyMap() {
// given
doReturn(objectMap).when(testObject).getMap();
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getMap),
Optional.of(objectMap));
}
@Test
public void testGetNotNull() {
// given
doReturn(object).when(testObject).getObject();
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getObject),
Optional.of(object));
}
@Test
public void testGetNotNullWhenNull() {
// then
assertEquals(abstractAccess.get(
null,
TestObject::getObject),
Optional.empty());
}
@Test
public void testNoneMatch() {
// then
assertTrue(abstractAccess.noneMatch(
ImmutableList.of(testObject),
o -> false));
}
@Test
public void testNoneMatchEmpty() {
// then
assertFalse(abstractAccess.noneMatch(
ImmutableList.of(),
o -> false));
}
@Test
public void testNoneMatchFalse() {
// then
assertFalse(abstractAccess.noneMatch(
ImmutableList.of(testObject),
o -> true));
}
}

WalmartLabs

Using technology, data and design to change the way the world shops. Learn more about us - http://walmartlabs.com/

Orren Chapman

Written by

Software Developer, Global eCommerce U.S.

WalmartLabs

Using technology, data and design to change the way the world shops. Learn more about us - http://walmartlabs.com/

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade