Consumer Driven Contract Testing for gRPC + Pact.io
Why Consumer Driven Contract Testing
Microservices come with great benefits: they can be self contained, loosely coupled, more maintainable and testable, independently provisioned and deployed, with distributed development and continuous delivery…
But they also come with some important challenges of its own: API Integration Testing and API Evolution Management being two of them.
API Integration tests is about how service integrations are tested:
- End-to-End tests with real environments can be the most reliable but difficult and expensive to setup, when not impossible.
- Mock Environments (docker, test containers…) are easier to setup but can quickly become out of sync, becoming harder to maintain and thus unreliable.
- Mocks like mockito are very easy to setup but injected responses can be very unreliable in evolving systems, as they may end having little to do with actual responses.
API Evolution & Breaking changes:
- How do we know when a service provider and its consumer implementations become incompatible
- When do we know it
These two questions are directly related with the technology we are using to implement and define our APIs as well as how we are testing them.
When using gRPC (and Protobuf) we can rely on the parser and code generation to maintain backward/forward compatibility between different versions of the same API but that is only half the problem.
For instance with Protobuf v3 all fields are optional, introducing a new field on the provider, outdated clients and server can still communicate but the common understanding or business logic may be definitely broken. It’s easy to see that updating the API definition on the clients will not be enough if business implementation it’s not also updated accordingly.
Contract Testing is not only about API compatibility and detecting breaking changes, it is also about independent but coordinated integration testing for both consumers and provider.
Contract testing with Pact.io allows you to define that shared understanding on a common document (“contract”) that can be used to test, independently but coordinated, that both provider and consumers are compatible.
From Pact.io website we can read:
Contract testing is a technique for testing an integration point by checking each application in isolation to ensure the messages it sends or receives conform to a shared understanding that is documented in a “contract”.
With Pact.io consumers can define in a simple DSL how they plan to interact with a given service, defining their requests and the shape of responses they expect to receive.
Pact.io will generate a “pact.json” file that will be used to:
- Start a mocking server for consumers to do integrations tests
- Replay requests and validate responses on the provider side
Pact.io is great to test REST APIs (or HTTP+JSON in general) but currently it does not have out-of-the-box support for technologies that use other protocols and serialization like gRPC that uses HTTP/2 and Protocol Buffers. For bridging that gap we have implemented an easy solution using only Java and Spring-Boot to make Pact.io+gRPC work together.
Consumer Driven Contract Testing with Pact + gRPC
While gRPC is commonly used with Protobuf over HTTP2, Pact.io works great to test HTTP + JSON APIs so we will need to proxy and transform gRPC requests in order to use Pact.io.
We have successfully implemented a simple solution to use Pact + gRPC in javaland with Spring-Boot, with no extra software or dependencies required, no extra open ports and absolutely no fuss.
Technology stack:
- Spring-Boot with LogNet grpc-spring-boot-starter (for provider implementation)
- pact-jvm-provider-spring: JUnit Test Runner (for provider verification)
- pact-jvm-consumer-java8: JUnit Test Runner (for consumer tests)
The following instructions are based on this stack but you can probably leverage this ideas to port it to your platform of choice.
Pact + gRPC with Java and Spring-Boot
For Pact and gRPC to work together we are going to need a way to transform Protobuf to JSON and connect gRPC to HTTP/1.1 back and forth.
On the consumer side:
- We are going to express our pacts as if request or response were JSON, no difference here.
- On consumer JUnit Tests, because pact MockHttpServer listens for HTTP requests, we will write a custom io.grpc.Channel and a io.grpc.MethodDescriptor.Marshaller for gRPC stubs to communicate with the mock server using http + json.
And this will be all we need on consumer side.
On the provider side:
- Because Pacts is going to replay all pact stored on the Pact-Broker for this provider as HTTP requests we are going to write an SpringMVC controller that will be available (auto scanned) only on tests, this controller will transform json requests into protobuf objects and call a gRPC in process server.
If your spring-boot application already listens on port 8080 this means no extra open port will be required for this to work.
- Listening to GRpcServerInitializedEvent spring-boot event we can collect all MethodDescriptor information, storing them by fullname we can latter use that information to map http urls to gRPC calls and marshall/unmarshall Protobuf to JSON.
In the following section you can find all implementation details you need to use Pact+gRPC for consumer driven contract testing.
Pact + gRPC implementation details and example
These will be all java classes (in bold) your need to implement this solution along with some sample classes to illustrate this example:
- product.proto: this is just a minimum example proto definition
- FindProductContractTest: this is just an standard pact-jvm JUnit test
- GrpcHttpChannel: this is where the magic happens on consumer side
- ProviderPactVerifyCDC: this is just an standard SpringRestPactRunner test class.
- GrpcHttpProxyController: this is where the magic happens on the provider verification side.
GRPC example definition
We are going to use the following proto definition. This is just minimum example:
GRPC <-> HTTP+JSON Transformation Convention used
GRPC methods will be mapped to HTTP:
- POST
- to url: “/grpc/” + grcp method fullname (package, service and method)
- content-type: application/json
- body: json payload (marshalled from protobuf)
Consumer Tests
The following class is just a standard pact-jvm JUnit test:
- The “contract” definition starts in line 23
- Line 56 defines a PactProviderRule junit rule that will start an http mocking server
- In line 46 we can find the actual test with grpc stub as client.
The only peculiarity on this class is that we are testing a gRPC stub (line 47) using a custom io.grpc.Channel (line 48) that will connect to pact http mocking server.
Proxying GRPC to HTTP mock server
We will connect gRPC calls to Pact mocking server to test our consumer.
Inside Channel.newCall(…) (L53) we will receive all information we need to connect to pact mocking server:
- gRPC method fullname to build http url (L96).
- Original request and response marshallers (L57–58).
- Request and response payload types and default values (L57–58).
- Json to protobuf marshallers (L196) based on default values.
Provider Verification Tests
This is just an standard SpringRestPactRunner there is nothing really special here, just notice how you can pass authentication headers to service (line 51), that can later be forwarded to gRPC process.
Proxing HTTP pact requests to GRPC service
This is where magic happens on the provider side:
We are just implementing an SpringMVC controller that can be placed in src/test/java so it’s only loaded for tests.
All information we need to map http to gRPC is collected on @EventListener in line 161 where we read all MethodDescriptor to be used later when handling incoming http requests.
Notice how we forward JWT credentials downstream to gRPC service.
Conclusion
While gRPC and Protobuf can define an “strongly typed” API “contract” and we can also rely on the parser and code generation to maintain backward/forward compatibility between different versions of the same API that is only half the problem.
Contract Testing can help you test your API shared meaning independently and coordinated while avoiding expensive end-to-end setups.
Contract Testing brings reliability at low cost in your service integration tests.
Contract Tests can replace service integration tests that ensure your provider and consumers communicate correctly while maintaining a common understanding. But they don’t replace functional tests that ensure that the core business logic of your services is working.
Now, with the ideas shared on this post you can leverage Pact.io capabilities and Contract Testing benefits also for your gRPC services.