Contract Tests With Pact JVM — The Tricky Parts
Contract testing is a great technique that provides a reliable way to verify the agreed boundaries between your microservices without all the complexity of integration testing.
It has some steep learning curve though. The following post covers some of the technical challenges my team had to overcome when introducing the Pact-based contract testing into our Java microservices world.
Which DSL variant shall I use for my Java project?
Java, Java 8 with lambdas, or Groovy DSL — which one is the best? That’s the exact question we asked ourselves before jumping into the world of pacts.
Let’s have a look at an example of JSON structure specified using different JVM DSLs. I’m going to use a product definition you might find in a typical warehouse system, with the following fields:
productId— a string value using UUID formatdimensions— a complex object containing decimal values of:width,length, andheightalternatives— an array of complex objects, each containing aproductIdand an integer value ofcount
Below you can find three code excerpts, limited to body matchers only, using different DSL variants: groovy, java, and java 8 (with lambdas):
For my team code readability was a key factor when selecting the DSL to define our contracts. Groovy DSL clearly stands out of the crowd here— it’s very concise and reflects the json structure without any additional noise. Some people might complain that it comes with a great learning curve though. However, considering the popularity of the Gradle build tool and Spock test framework, it’s very unlikely that Groovy is going to be something completely new to everyone in your team. Even if it is, it’s going to be a great opportunity to learn and make use of it in other contexts!
The classic Java DSL is, as mentioned by the official Pact documentation, error-prone and hard to read. The need to explicitly open (and close) every element — be it an object or an array — makes it very difficult to follow the levels of the json structure. Another serious drawback is that the DSL doesn’t stop you from making mistakes, as some methods can be used in a specific context only, but are always available. On last word about the style — it’s not very aligned when it comes to method naming — see stringType(...) and uuid(...) matcher methods — which may make it a bit more difficult to understand.
Even though Java 8 DSL with lambdas seems to be a major improvement over the classic Java DSL, its readability is still far worse compared to Groovy DSL. Having lambdas instead of opening and closing different elements is quite concise, but requires a lot of boilerplate code anyways. Even if you name your lambda parameters using a single letter only, it’s still a lot of arrows and brackets everywhere. There’s also a small, but quite annoying problem to it — the indentation — as IDEs tend to remove the additional whitespace that could indicate the level in json structure, making everything look like it’s at the root level. The good thing is that it comes with the good old static typing — which may be the reason for some people to prefer it over Groovy DSL.
Getting groovy on the right path
Great, let’s say you’ve just selected Groovy DSL for Pact JVM as suggested above, and moved to configuration. After the initial setup of dependencies in your Gradle project, you created a draft SomeConsumerContractTest.groovy class. Then you ran Gradle build, and boom, all you get is the following failure in your console:
Execution failed for task ':compileTestGroovy'.
> Could not resolve all files for configuration ':detachedConfiguration1'.
> Could not find org.codehaus.groovy:groovy:4.1.0.You might wonder, what is detachedConfiguration1 or why is Gradle looking for some weird version — 4.1.0 — that’s exactly the same as of Pact’s groovy consumer?
It turns out there’s a naming clash between Gradle groovy plugin and Pact’s groovy consumer that leads to mysterious errors like this. It all boils down to how Gradle groovy plugin handles automatic configuration of groovyClasspath — in a nutshell, it looks for an artifact named groovy or groovy-all, without taking the group name into consideration…
How to avoid it?
There’s a straightforward fix for it — you need to declare the groovyClasspath yourself to avoid the automatic configuration, i.e.:
Hardcoded vs generated values
This one is more about the right approach rather than technical aspects of a selected Pact DSL.
When defining a consumer-driven contract it’s recommended to focus on the request or response structure and avoid exact values matching.
So, instead of putting up a contract with hardcoded values of Tom in a request and Hello Tom! in a response:
You, as an API consumer, should rather focus on data types:
However that’s only half the story. Let’s consider a provider perspective now. When verifying your consumer contracts you need to set up external dependencies properly — i.e. stub some responses from services you depend on.
With a pact written as in Example B you’ll always get some randomly generated String in a name field of a POST request to your /hello endpoint. That requires very liberal, generic mocking on provider side, something like:
Even though it may be a sufficient approach in simple cases, it becomes tedious for a provider, especially when there are multiple consumers of his API that require slightly different responses depending on the name parameter.
What’s the recommended approach?
Use Pact’s DSL matchers that can take an example as an argument! Define your contract like this:
Notice the usage of string('Tom') in a request, and string('Hello Tom!') in a response. They are useful for both sides of a pact — a consumer and provider. Let’s find out how they can make the most out of it.
Provider’s perspective
Provider is going to verify if its responses fulfill the Consumer needs and match against the type string in the response (❶). What’s more, the example string value 'Tom' (❷) from the request may be used when mocking external dependencies in a dedicated test class for verifying contracts on provider side, i.e.:
Consumer’s perspective
When creating the consumer-driven contract there’s a promise to the provider that any requests sent to it will match against the type string (❸). This will be verified in a test generating the pact in Consumer’s codebase. Additionally, the string value 'Hello Tom!' (❹) may be used when running and verifying the pact and its generated response, i.e.:
Heterogeneous collections matching
At first, let’s consider the following JSON response example:
The items array may contain items of two types: a regular item with a price, and a discounted item that has a discounted price only. They both share some common fields: id and type. The array is heterogeneous, meaning its elements’ structure may vary.
Some people consider it as an example of bad API design. In fact, according to Pact’s philosophy, it is expected that arrays are homogeneous — their elements have always the same structure. Hence the eachLike() and other matchers for you to verify array contents in your contracts.
As for now with Pact standard ver. 3, there’s no support for array matchers that would work as: match if array element matches any of the specified structures.
Even though it may be viewed as a badly designed API, heterogeneous arrays may appear in your contract tests from time to time. How to handle it with Pact?
Mixed approach — specific elements with matchers
One approach would be to define a contract as if our array was homogeneous, meaning including all the fields from both types of items together, i.e:
The approach above will, however, generate a response containing an item with a structure that’s invalid from both business and structural point of view — we shouldn’t see a regular item with discounted price.
Whereas in some cases it may be enough to test a typical structure and leave the less common ones to component tests I believe there’s a better approach to it. Consider the following contract definition:
Please mind there are some trade offs here. Firstly, we sacrifice the “avoid exact value matching” principle in order to make sure we cover all of the expected structures in our contract tests.
Secondly, having items defined with an exact number of variants puts additional effort on the producer. It wouldn’t be possible to be liberal about the generated response when verifying consumer expectations on producer side anymore, but rather require exactly two items in the array, ordered in a way to match the expected structures — first a regular item and then a discounted one.
Wrap up
If you’re in a Java team thinking about introducing Pact-based contract tests — just give it a go! Use Groovy DSL, try to provide your type matchers with some examples wherever possible, and most importantly, do not follow all the Pact principles too rigorously.
