Java: On The Benefits of Treating DTOs as Magic Cookies
An examination of complex data handling and robust Java application development.
There’s nothing magical about data transfer objects (DTOs); in their purest Java language form they contain a set of typed data fields with no responsibilities beyond aggregation. Treated as the simplest form of JavaBeans, DTO methods consist of getters and setters for their fields, with perhaps customized equals, hashCode and toString implementations.
Magic cookies in computer science parlance, are even simpler in purpose than DTOs: magic cookies are opaque blobs of data with no fields, no getters and no setters. While virtually meaningless in one context, they presumably are quite meaningful in another. As such, magic cookies are a kind of split-personality DTO: no meaning here, lots of meaning somewhere else.
Problem Statement
Within a distributed application architecture, DTOs carry the information between the application tiers. Because of the resource costliness of each individual communication between those tiers (e.g., latency, bandwidth), the resulting DTOs are often packed with lots of data representing complex information models. These models may be highly hierarchical in structure, a.k.a., “deep”: e.g., a set of things containing one or more lists of other things that contain even more things, on and on.
To address contextualized business inquiries (e.g., is this product transactable online?) may require traversal of many layers of these DTO hierarchies. And, given the nature of time based transactions, at each stage of those traversals, the next level in the hierarchy may be incomplete in whole or part. When working in Java these traversals if not carefully programmed can result in runtime exceptions; e.g., a NullPointerException.
Unit tests are a strategy to minimize application implementation risks; these tests require input data that will exercise the critical pathways in the application code and the broad range of possible inputs that may occur in production environments. Constructing and maintaining these complicated DTO hierarchy inputs very quickly becomes cumbersome and leads to complex test setups exceeding that of the application code itself; and, which may well obscure any assurance that the test results actually provide thorough application code coverage.
Example Code
Object Alpha has a field beta that has a field gamma which has a String field containing a description. It's a simple DTO hierarchy and far more complex things exist in nature.
public class Alpha {
private Beta beta; public Beta getBeta() {
return beta;
} public void setBeta(Beta beta) {
this.beta = beta;
}
}public class Beta {
private Gamma gamma; public Gamma getGamma() {
return gamma;
} public void setGamma(Gamma gamma) {
this.gamma = gamma;
}
}public class Gamma {
private String description; public String getDescription() {
return description;
} public void setDescription(String description) {
this.description = description;
}
}
Then the “safe” or “bulletproof” Java code implementation to access the description field given the alpha input object would look something like:
if (alpha != null &&
alpha.getBeta() != null &&
alpha.getBeta().getGamma() != null &&
alpha.getBeta().getGamma().getDescription() != null) {
// ...
}Some Common [Painful] Unit Test Solutions
Common strategies for unit testing applications accessing complex DTO hierarchies include mocking the test data hierarchy and assembling an inventory of canned test data inputs. Both these solutions succeed in the purpose but at considerable expense in terms of implementation and maintenance and both fail to provide the assurance of truly comprehensive coverage.
Mocked Test Data
Mocking complex DTOs provides a versatile approach for verifying just about any application test scenario. Building out these mocked DTOs may require a great deal of test setup code in itself. And, as the complexity of the DTO hierarchy increases, the number of test scenarios likewise increases.
Example Code
@Mock
private Alpha alpha;
@Mock
private Beta beta;
@Mock
private Gamma gamma;@BeforeMethod
public void setUp() {
initMocks(this); doReturn(beta).when(alpha).getBeta();
doReturn(gamma).when(beta).getGamma();
doReturn("description").when(gamma.getDescription());
}
That covers the happy path, but, one would have to account for the other scenarios: alpha is null, beta is null, gamma is null, or description is null.
Canned Test Data
Preserving DTOs in serialized forms (e.g., JSON) to be loaded as test inputs limits the amount of test setup code that mocking requires. This approach instead requires identifying and preserving the test inputs and then maintaining them overtime. When the data inputs are complicated these serialized versions may be quite large; and, as more scenarios are identified many copies may need to be maintained.
Example Code
JSON found in alpha.json.
{
"beta": {
"gamma": {
"description": "..."
}
}
}Once again, this covers the “happy path” scenario.
A Magic Cookie Approach
By treating DTOs as opaque things, i.e., magic cookies, solves both the complexity of accessing data and its evaluation. The following steps describe how.
Step 1: Separation of Concerns: Access v. Evaluation
Separating the logic surrounding accessing data from evaluating the results of that access, provides a means of comprehensively verifying these two different types of logic independently of one another. Access functionality then is encapsulated in a separate class in which the methods implement the different types of access on the corresponding DTO type.
Example Code
An AlphaAccess class models the types of access on the Alpha "magic cookie". It is a singleton.
public class AlphaAccess {
private static final AlphaAccess INSTANCE = new AlphaAccess(); protected AlphaAccess() {
} public static AlphaAccess getInstance() {
return INSTANCE;
} public Beta getBeta(Alpha alpha) {
return alpha != null ? alpha.getBeta() : null;
}
}
Step 2: Modeling “Safe” Access
An Access class method that returns a null value doesn’t mitigate the risk of programmer error when accessing data. Java 8 introduced the Optional class that can be used to eliminate null return values on Access methods.
Example Code
The updated version of the getBeta method implemented using the Optional return value:
public Optional<Beta> getBeta(Alpha alpha) {
return alpha != null ?
Optional.ofNullable(alpha.getBeta()) :
Optional.empty();
}Step 3: Refactoring Access: The Blockbuster Superclass
Access methods are “go-fer” style methods that thanks to Java 8 Method References, can be greatly simplified using an abstract superclass:
Example Code
An abstract superclass of a specific DTO Access class provides a common implementation to “get” a field value:
public abstract class AbstractAccess<T> { protected <R> Optional<R> get(T o, Function<T, R> getter) {
return o != null ?
Optional.ofNullable(getter.apply(o)) :
Optional.empty();
}
}
And the AlphaAccess subclass is then implemented as:
public class AlphaAccess extends AbstractAccess<Alpha> {
... public Optional<Beta> getBeta(Alpha alpha) {
return get(alpha, Alpha::getBeta);
}
}
And, given a separate unit test for the AbstractAccess class, the unit test for the AlphaAccess class is straightforward, too:
public class AlphaAccessTest {
private AlphaAccess alphaAccess;
@Mock
private Alpha alpha;
@Mock
private Beta beta; @BeforeMethod
public void setUp() {
initMocks(this);
alphaAccess = new AlphaAccess();
} @Test
public void testGetBeta() {
// given
doReturn(beta).when(alpha).getBeta();
// when
Optional<Beta> actual = alphaAccess.getBeta(alpha);
// then
assertEquals(actual, Optional.of(beta));
}
}
This provides 100% code coverage for the access logic contained within AlphaAccess.
Step 4: Delegating Access: Enforcing Scope
Having created access methods on top-level fields, additional methods for fields accessed via those next level objects can be created at this top level, too, simplifying application access and verification code.
Example Code
AlphaAccess has a new injected dependency, BetaAccess, to which it will delegate responsibilities for its data, and, BetaAccess, too, has an injected dependency, GammaAccess, to which it will delegate responsibility for access to its description field:
private static final AlphaAccess INSTANCE =
new AlphaAccess(BetaAccess.getInstance());// injected dependency
private final BetaAccess betaAccess;protected AlphaAccess(BetaAccess betaAccess) {
this.betaAccess = betaAccess;
}...public Optional<String> getDescription(Alpha alpha) {
return getBeta(alpha).flatMap(betaAccess::getDescription);
}
In BetaAccess has a similar implementation:
public Optional<String> getDescription(Beta beta) {
return getGamma(beta).flatMap(gammaAccess::getDescription);
}And, finally in GammaAccess we have:
public Optional<String> getDescription(Gamma gamma) {
return get(gamma, Gamma::getDescription);
}And, now the unit test for AlphaAccess would include:
@Test
public void testGetDescription() {
// given
doReturn(beta).when(alpha).getBeta();
doReturn(Optional.of("description")).when(betaAccess)
.getDescription(beta);
// when
Optional<String> actual = alphaAccess.getDescription(alpha);
// then
assertEquals(actual, Optional.of("description"));
}Note, that betaAccess is a mocked object of type BetaAccess, injected at AlphaAccess construction and stubbed here to return an Optional object of the description field “description” String value. This stubbing is not unit test cheating in that the behavior is outside the scope of the AlphaAccess class itself whose only responsibility is to retrieve the beta object and delegate to its Access class, BetaAccess, the responsibility of obtaining the Optional object containing the description field's value.
Step 5: The Big Payoff: Do Something If a Value Is Present
Now, our application code can safely implement its function when a value is present and without incorporating all the complexities associated to obtaining that value.
Example Code
The Optional object enables NullPointerException-free access code.
public class App {
// injected dependency
private final AlphaAccess alphaAccess; private final Alpha alpha; public App(Alpha alpha) {
this(AlphaAccess.getInstance(), alpha);
} protected App(AlphaAccess alphaAccess, Alpha alpha) {
this.alphaAccess = alphaAccess;
this.alpha = alpha;
} public void doSomething() {
alphaAccess.getDescription(alpha).ifPresent(description -> {
// ...
});
}
}
And the application unit test then is simplified because only the alpha “magic cookie” and its corresponding access class need be mocked and stubbed.
public class AppTest {
private App app; @Mock
private AlphaAccess alphaAccess;
@Mock
private Alpha alpha; @BeforeMethod
public void setUp() {
initMocks(this); app = new App(alphaAccess, alpha);
} @Test
public void testDoSomething() {
// given
doReturn(Optional.of("description")).when(alphaAccess)
.getDescription(alpha);
// when
app.doSomething();
// then ...
}
}
Achieving 100% application code coverage is much more readily achievable now that the AlphaAccess class encapsulates the underlying details of how the description field value was retrieved; the unit test can focus on the evaluation of the results instead of the complexity of building out the complex DTO inputs.
Step 6: Postscript: Filtering the Results
It maybe useful to universally treat empty Collection subclasses and Map implementations and blank String objects as missing (Optional.empty()). This can be accomplished easily in the Access superclass.
Example Code
The AbstractAccess class is updated to filter out empty Collection and Map instances and blank String instances:
public abstract class AbstractAccess<T> { protected <R> Optional<R> get(T o, Function<T, R> getter) {
return o != null ?
Optional.ofNullable(getter.apply(o))
.filter(r -> !(r instanceof String) ||
StringUtils.isNotBlank((String) r))
.filter(r -> !(r instanceof Collection) ||
CollectionUtils
.isNotEmpty((Collection<?>) r))
.filter(r -> !(r instanceof Map) ||
MapUtils.isNotEmpty((Map<?, ?>) r)) :
Optional.empty();
}
}
Step 7: Postscript: Additional Collection Support
Finally, it is often useful to support different Collection-oriented filters. Testing whether all members of a Collection match a particular criteria (Predicate) or any do or none do, finding one that matches or finding the first one that matches
Example Code
Once again the AbstractAccess superclass is updated to provide support for different type of Stream-oriented access types: allMatch, anyMatch, findAny, findFirst, noneMatch.
public abstract class AbstractAccess<T> { protected boolean allMatch(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) &&
collection.stream().allMatch(predicate);
} protected boolean anyMatch(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) &&
collection.stream().anyMatch(predicate);
} protected Optional<T> findAny(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) ?
collection.stream().filter(predicate).findAny() :
Optional.empty();
} protected Optional<T> findFirst(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) ?
collection.stream().filter(predicate).findFirst() :
Optional.empty();
} protected <R> Optional<R> get(T o, Function<T, R> getter) {
// ...
} protected boolean noneMatch(Collection<T> collection,
Predicate<T> predicate) {
return isNotEmpty(collection) &&
collection.stream().noneMatch(predicate);
}
}
Conclusion: Better Applications
By separating data access logic from data evaluation logic we are able to implement more reliable and maintainable software. The complexity of the data model no longer obscures the complexity of the application itself. And, verifying the two types of logic in isolation allows for a comprehensive examination of the important scenarios that need to be addressed in both domains, access and evaluation.
Next Steps: Java 10 Extensions
With Java 10, the Optional class has some important extensions which will be very useful in conjunction with the Access class strategy described here; specifically the ifPresentOrElse, or and stream methods will help data evaluation code address common scenarios.
Example Code: Optional#ifPresentOrElse(Consumer, Runnable)
Some application code where the absence of a particular value is as significant as its presence.
alphaAccess.getDescription(alpha).ifPresentOrElse(description -> {
// do something with the description ...
}, () -> {
// do something when there is no description ...
});Example Code: Optional#or(Supplier)
Some data scenarios have alternative resolution strategies which may be encapsulated within the Access classes themselves. In the case of the Gamma class with two fields, description and backupDescription.
public class Gamma {
private String description;
private String backupDescription; // ...
}
If the rule is: when the description field is blank then the backupDescription field should be used; then, the GammaAccess class might encapsulate this rule by implementing:
public Optional<String> getDescription(Gamma gamma) {
return get(gamma, Gamma::getDescription)
.or(() -> get(gamma, Gamma::getBackupDescription));
}Example Code: Optional#stream()
When dealing with collections, we often must deal with a Stream of Optionals, Java 10 helps simplify by dereferencing the non-empty instances:
Stream<Optional<Alpha>> alphaStream = ...;
List<String> descriptions =
alphaStream
.flatMap(Optional::stream)
.map(alphaAccess::getDescription)
.flatMap(Optional::stream)
.collect(Collectors.toList());Appendix: AbstractAccess Unit Test
Here’s the source for the AbstractAccess super class which provides 100% code coverage for its helper methods so that the Access subclasses can focus for the most part on just their own "happy path" scenarios.
import java.util.List;
import java.util.Map;
import java.util.Optional;import org.mockito.Mock;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;import com.google.common.collect.ImmutableList;
import com.walmart.terrene.TerreneTestCase;import static org.mockito.Mockito.doReturn;
import static org.mockito.MockitoAnnotations.initMocks;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertFalse;
import static org.testng.Assert.assertTrue;public class AbstractAccessTest extends TerreneTestCase { protected interface TestObject {
List<Object> getList(); Map<Object, Object> getMap(); Object getObject(); String getString();
}
private static final String BLANK = "";
private static final String NOT_BLANK = "not-blank";
private AbstractAccess<TestObject> abstractAccess;
@Mock
private TestObject testObject;
@Mock
private Object object;
@Mock
private List<Object> objects;
@Mock
private Map<Object, Object> objectMap; @BeforeMethod
public void setUp() {
initMocks(this); abstractAccess = new AbstractAccess<TestObject>() {
};
} @Test
public void testAllMatch() {
// then
assertTrue(abstractAccess.allMatch(
ImmutableList.of(testObject),
o -> true));
} @Test
public void testAllMatchEmpty() {
// then
assertFalse(abstractAccess.allMatch(
ImmutableList.of(),
o -> true));
} @Test
public void testAllMatchFalse() {
// then
assertFalse(abstractAccess.allMatch(
ImmutableList.of(testObject),
o -> false));
} @Test
public void testAnyMatch() {
// then
assertTrue(abstractAccess.anyMatch(
ImmutableList.of(testObject),
o -> true));
} @Test
public void testAnyMatchFalse() {
// then
assertFalse(abstractAccess.anyMatch(
ImmutableList.of(testObject),
o -> false));
} @Test
public void testAnyMatchNull() {
// then
assertFalse(abstractAccess.anyMatch(
null,
o -> true));
} @Test
public void testFindAny() {
// then
assertEquals(abstractAccess.findAny(
ImmutableList.of(testObject),
o -> true),
Optional.of(testObject));
} @Test
public void testFindAnyEmpty() {
// then
assertEquals(abstractAccess.findAny(
null,
o -> true),
Optional.empty());
} @Test
public void testFindFirst() {
// then
assertEquals(abstractAccess.findFirst(
ImmutableList.of(testObject),
o -> true),
Optional.of(testObject));
} @Test
public void testFindFirstEmpty() {
// then
assertEquals(abstractAccess.findFirst(
null,
o -> true),
Optional.empty());
} @Test
public void testGetBlank() {
// given
doReturn(BLANK).when(testObject).getString(); // then
assertEquals(abstractAccess.get(
testObject,
TestObject::getString),
Optional.empty());
} @Test
public void testGetEmptyMap() {
// then
assertEquals(abstractAccess.get(
testObject,
TestObject::getMap),
Optional.empty());
} @Test
public void testGetNotBlank() {
// given
doReturn(NOT_BLANK).when(testObject).getString(); // then
assertEquals(abstractAccess.get(
testObject,
TestObject::getString),
Optional.of(NOT_BLANK));
} @Test
public void testGetNotEmptyList() {
// given
doReturn(objects).when(testObject).getList(); // then
assertEquals(abstractAccess.get(
testObject,
TestObject::getList),
Optional.of(objects));
} @Test
public void testGetNotEmptyMap() {
// given
doReturn(objectMap).when(testObject).getMap(); // then
assertEquals(abstractAccess.get(
testObject,
TestObject::getMap),
Optional.of(objectMap));
} @Test
public void testGetNotNull() {
// given
doReturn(object).when(testObject).getObject(); // then
assertEquals(abstractAccess.get(
testObject,
TestObject::getObject),
Optional.of(object));
} @Test
public void testGetNotNullWhenNull() {
// then
assertEquals(abstractAccess.get(
null,
TestObject::getObject),
Optional.empty());
} @Test
public void testNoneMatch() {
// then
assertTrue(abstractAccess.noneMatch(
ImmutableList.of(testObject),
o -> false));
} @Test
public void testNoneMatchEmpty() {
// then
assertFalse(abstractAccess.noneMatch(
ImmutableList.of(),
o -> false));
} @Test
public void testNoneMatchFalse() {
// then
assertFalse(abstractAccess.noneMatch(
ImmutableList.of(testObject),
o -> true));
}
}
