Pact and Contract Testing as Part of a Complete Testing Strategy
Contract testing and Pact, they’re Gr-r-reat!
Contract testing is a popular topic at software engineering conferences and talks. The pitch for contract testing is that you:
- Define a contract between two microservices.
- Test that contract against each microservice.
- Only need to run one microservice at a time.
The resource requirements for contract testing are an order of magnitude less than spinning up the world to run integration tests on a software system with multiple microservices.
Contract testing sounds interesting, but how could it be applied to a testing strategy?
To answer this, let’s start with an example system and add contract testing with Pact.
The System
Consider a system with multiple microservices. How is it tested today?
Start with a single empty microservice.
Add first class, then test
Unit Tests perform white box testing on the methods in a class.
Unit Tests:
- Create instances of the class being tested.
- Create mocks for external classes and external resources that are dependencies of the class.
- Are lightweight and easy to run. Little to no manual effort is needed to run a test in an IDE or with Gradle.
Add many classes, then test
Integration Tests perform black box testing on the endpoints of a microservice.
Microservice Integration Tests:
- Assume a running instance of the microservice. Generally there is manual coordination between starting a running local instance and starting the integration tests.
- Create mocks for external microservices and resources that are dependencies of the microservice.
- Test the combined behavior of the classes in the microservice.
- Do not retest what was already validated by unit tests.
- Require more resources to run, but still generally fit on a developer machine.
Add many microservices, then test
Integration Tests could also perform black box testing on a complete system.
System Integration Tests:
- Have no external microservices and resources. Everything is spun up
- Test the combined behavior of the microservices in the system.
- Do not retest what was already tested by microservice integration tests. This leaves validating messages and behavior that was previously mocked as external systems.
- Assume a running instance of the system. The system is complex to setup, manually started, and shared between developers.
Summary about testing today
- Unit Tests and Microservice Integration tests validate a system on a development machine pretty easily, and only consume resources when testing.
- System integration tests require a large external staging environment that is difficult to set up, must be shared between developers, consumes resources 24x7, and requires external access.
Adding Pact
Contract testing is a popular topic at software engineering conferences and talks. Pact is frequently mentioned as a popular tool to use for contract testing. The Pact name is a synonym of the word contract.
Let’s dig into Pact and look at the mechanics to creating a contract test.
Pact Definitions
- In Pact, a pair of services that exchange messages is called a consumer and a provider.
- State adds conditions that change which messages are valid: user logged in, order completed, etc.
- An interaction is a request and response between a consumer and a provider, optionally restricted to a given state.
- Interactions are not behavior. Correct behavior should be isolated to a single microservice and tested with microservice integration tests.
- A contract test validates what happens before and after an interaction.
- A set of interactions is stored in a Pact file.
- A Pact Broker is a repository service for Pact Files organized by provider name.
- A service mock is mocked at the HTTP layer
Steps to using Pact
- Unit tests should already be mocking external services. Use Pact to define interactions inside consumer unit tests. Pact will use interactions to generate replacement service mocks.
- Adapt unit tests to use the Service Mocks generated by Pact.
- Now when running unit tests, in addition to testing against Service Mocks, Pact will also record all interactions into a pact file.
- Copy the pact file to a pact broker.
- Instruct Pact to generate integration tests from the pact broker and run them against a running instance of each provider microservice.
Partial Pact Has Value
Even if setting up a Pact broker and automating provider testing is too much work today, select Pact (over other wire mock solutions) for mocking external services in consumer tests. When there is time to complete the other half of contract testing, the work of writing Pact consumer tests will already be complete.
Pact is Polyglot — Java Example
There are Pact implementations in many languages. This goal of this section is not intended to teach how to write contract tests with Pact, but to give one expectation of what tests may look like. This example uses the pact-jvm-consumer library:
Pact Definition
@Pact(provider="myRequestProvider", consumer="myServiceConsumer")
public RequestResponsePact createPact(PactDslWithProvider builder) throws JsonProcessingException {
return builder
.uponReceiving("Example MyRequest POST")
.path("/my-service/serviceA.json")
.body(objectMapper.writeValueAsString(new MyRequest()))
.method("POST")
.willRespondWith()
.status(200)
.body(objectMapper.writeValueAsString(new MyResponse()))
.toPact();
}Pact Test
@Test
@PactTestFor(providerName = "myRequestProvider")
void test(MockServer mockServer) throws IOException {
Response response =
Request
.Post(mockServer.getUrl() + "/my-service/serviceA.json")
.bodyString(
objectMapper.writeValueAsString(new MyRequest()),
ContentType.APPLICATION_JSON)
.execute();
HttpResponse httpResponse = response.returnResponse(); assertEquals(200,
httpResponse.getStatusLine().getStatusCode()));
assertEquals(new MyResponse(),
objectMapper.readValue(
EntityUtils.toByteArray(httpResponse.getEntity()),
MyResponse.class));
}
}
Pact Summary — Pact is Lightweight
There are three steps to adding Pact to a microservice system:
- For each consumer microservice, use Pact for wire mocks.
- Install a Pact broker once and share it among all microservices. Unit tests will now update Pact broker while they validate client side interactions.
- For each provider microservice, ask Pact to generate and run integration tests every time integration tests are run to validate server side interactions.
That’s it! Welcome to contract testing.
Contract testing compared with other testing
Unit Tests and Microservice Integration Tests are great for testing most of a system, but they do not test:
- Shared behavior between microservices.
- Messaging formats.
Shared Behavior
Shared behavior is an anti pattern and hard to test. It is validating that an external service would return what a mock of that service is returning. Without shared behavior, only one microservice in an interaction:
- Knows what is the correct behavior is.
- Tests the behavior correctness with unit tests and microservice integration tests.
For example, if a system only allows five payment types, that fact should be enforced by a single microservice. Other microservices should depend on that single microservice for the knowledge about payment types.
But, if another microservice were also given knowledge of valid payment types (shared behavior), both microservices would now need to be tested together to validate they support the same payment types. Testing two microservices with shared behavior would need to be coordinated, either by running both systems at the same time (doubling resource requirements), or crafting a new system to describe and pass valid payment types from the first system test to the second system test (creating a new layer of complexity).
Messaging Formats
After validating each microservice, and eliminating shared behavior, the only remaining thing left to test in the system are message formats:
- Request format
- Response formats (Happy, Sad, others)
Message formats are contracts, literally. Contract testing is the final piece in the system testing strategy.
Wrapping it all up
As promised, contract testing tests one of the two things we use a full system to test: message contracts.
But we still need a full system staging environment to validate shared behavior. The two options around this are:
- Fix your code to remove shared behavior.
- Continue to run integration tests in a full system in a staging environment.
A full system staging environment still has other potential uses uses, such as load testing or a production hot swap. But with contract testing, you have a path to reducing dependency on a staging environment and increasing automation around the development workflow.
