PACT — Enabling Contract Test Automation

source: https://www.google.com

In my last blog on contract testing fundamentals, we learnt the concept of consumer driven contract testing and the value it brings for agile teams working in a microservices architecture. It’s now time to get down to the trenches of its practical implementation via programming. In this blog, we’ll learn implementing a tool for automated contract testing known as PACT.

PACT — defining the tool

PACT is an open source tool built to create & validate contracts between service providers and consumers. It allows defining interaction rules from the customer perspective and makes the provider verify them with every change it makes. The tool provides a group of libraries, available in various programming languages, that lets engineering team setup contract tests with an out-of-the box mock server. The tool manages compatibility matrices across the app versions of both parties(as they evolve) through a middleware known as Pact broker — an application responsible for keeping consumers and providers informed about the status of contract compliance(more on brokers a bit later in this blog).

PACT makes it very convenient for teams to manage contract testing through a rich library of verification centric annotations (thus avoiding a lot of boiler plate code) and seamless integrations with popular unit testing tools for quick test creation & executions.

PACT Workflow

PACT’s primary emphasis is on validating HTTP request-response structures. Using PACT, services that are communicating through APIs can get into an agreement basis certain blueprints shared by the provider(s). Consumers use these blueprints to create pacts with requests they would make and the response structure (headers, body, status codes) they expect (including only those fields that they would consume from the entire response).

source: https://www.google.com

PACT lets consumer create instance of a mock server(which acts as the provider) and run API tests against it. The success of these tests results into a contract file comprising of interactions that happened. Contract file is then shared with the provider.

source: https://www.google.com

Provider downloads the contract file and replays the same tests — send the same set of requests present in the contract file and receive the response, against the real service(actual provider). Contract is said to be verified if real responses are matching the rules defined by the contract, else the contract is said to be violated.

PACT Flow components

source: https://www.google.com

Consumer : The service application that is depending on some other service for retrieving some information

Provider : The service application that generates some information and provides to its dependents

Mock provider : Out-of-the box mock server available in the PACT library that allows creating pacts in the consumer code

Mock consumer : Requests present in the shared contract which are executed against the real service provider

Pact Unit Tests : Contract tests written in the consumer app that makes a request against the mock provider and asserts the mock response

Play & Record : The event of running unit tests and writing the successful interactions in a request-response format inside the contract file

Pact file : The contract file which is shared with the provider. It defines involved request-response structures and the matching rules

Replay & Verify : The event of re-running the requests recored in the pact file against the real provider and verifying the responses against matching rules in the pact file

The important aspect here, is the middleware that does an instrumental job of keeping consumers & providers in sync- the PACT broker. Let us dig inside the broker utilities below.

PACT Broker Application

A typical PACT broker UI

PACT broker is an open source application that does the following jobs in a PACT environment :

1. Enable consumers to publish contracts to a central repo
2. Enables providers to download contracts for replaying them
3. Enables providers to publish verification results back to the central repo
4. Maintains contract versions automatically
5. Manages matrix of verification results for each consumer-provider version
6. Provide web hooks to trigger event based actions
7. Expose APIs that identifies various provider versions and assist consumers take deployment decisions

PACT broker sorts the effort of managing multiple provider-consumer relationships and their contract verification matrices over a period of time. It is backed by a postgres DB for storing all the information.

PACT-JVM

Now that all components of the PACT setup are clear, let’s start coding!!

For the scope of this blog, we’ll focus on the java implementation of PACT, known as PACT-JVM.

Steps to implement PACT-JVM

Consumer side codebase

Pre-requisites:

1. Consumer pact library (gradle dependency : au.com.dius:pact-jvm-consumer-junit:4.0.3)
2. Plugin for publishing contracts (gradle plugin classpath : gradle.plugin.au.com.dius:pact-jvm-provider-gradle:4.0.10)
3. Any Rest API client
4. JUnit library

Actions :

1. Define pacts based on blueprint of the API

This Pact simply describes that when a request like this comes, server should respond with all body params as indicated & status code 200

2. Instantiate the in-built mock server

3. Automate the HTTP APIs using the REST client

4. Write unit tests for the automated APIs while consuming the pacts created in step 1

5. Run the tests against the mock server(treated as the real provider at this point)

6. Upon the success execution of tests, a JSON file is created in output/pacts folder -> that is the contract file

Small snippet from a contract file

7. Publish contract to the broker instance using the pact plugin (gradle command : ./gradlew pactPublish)

gradle task for publishing contract to the broker (Removed the host for privacy reasons)

Broker setup

1. Install a broker instance on a centrally accessible machine(cloud/VM etc) — highly recommended way is to go for a docker based multi container application(a composition of pact broker and a postgres app — reference)
2. Launch the broker host on browser and validate that the consumer-provider entry is made to its home page(refer the screenshot added in the PACT broker section above)

Provider side codebase

Pre-requisites:

1. JUnit
2. Provider Pact library (gradle dependency : compile group: 'au.com.dius', name: 'pact-jvm-provider-junit', version: '4.0.10')

Actions

1. Define certain system properties for connecting to PACT broker & publishing verification results

(Removed the host for privacy reasons)

2. Setup mock consumer and provide information on the real service to replay tests from the contract

(Removed the real service host for privacy reasons)

3. Execute the test class which would publish verification results back to the broker

Matrix screen on the broker UI for the consumer-provider integration

As the matrix screen makes it evident, contract verification is maintained across various consumer-provider versions and gives a clear picture of compatibility.

Note that the contract version is not explicitly mentioned anywhere, since it is maintained by the broker internally and is based on the consumer version itself. There could exist multiple consumer versions with the same contract version at a given point in time and verification job on the provider side would update verification results against each consumer version.

can-i-deploy tool

can-i-deploy is an API exposed by PACT broker which does the real magic — it compares verification results between various consumer and producer versions and notify consumer that whether it is safe to deploy a particular version or not.

This tool reads through all available results on the broker matrix for consumer-provider integration and ensures that there exist any successful verification for the particular consumer version against the tagged provider version(details on using the tool in the next section)

A successful can-i-deploy response looks like this :

On the other hand, a failed response looks like this :

The results makes it absolutely clear, doesn’t it!

Tips for adding PACT implementation to a CI/CD environment

After making sure that consumer pipeline is equipped with jobs for running the tests & publishing contract and provider pipeline has got the verification job, we shall implement the following stuff to make this flow completely automated

1. Create a webhook using PACT broker (API or the UI) that triggers automated execution of verification job on provider pipeline after the contract publish job on consumer pipeline passes

API example for creating a webhook using broker URL as the host and having a gitlab job to be triggered upon receiving the mentioned event

2. Execute a tag creation job on provider pipeline to associate provider version against an environment

Letting the broker know that the latest provider version is tagged as stagingEnv and the can-i-deploy tool should be using this tag for reaching to a decision (Removed the base URL for privacy reasons)

3. Trigger execution of can-i-deploy job on consumer pipeline (automated execution using PACT broker webhook or CI triggers)

Letting the consumer know that whether it can deploy this version to environment tagged at producer side(known to both upfront) as stagingEnv based on the matrix results(Removed the base URL for privacy reasons)

Conclusion

PACT acts as a comprehensive contract testing solution with its easy-to-setup tool set and closeness to the development codebase. It is highly recommend to use this tool for an end-to-end automated contract verification in large organisations striving to build big systems composed of multiple dependent components, without making any component wait for the other. With knowledge of such a tool, QA folks can build another bridge that would take them closer to development strategies and make them fully equipped for the relevant challenges.

Happy Testing!