Contract Testing with Pact

How to deal with a situation when the end-to-end tests are overloaded with test scenarios and their execution time increases over time? How to detect breaking changes in one of microservices API before deployment to production.

These questions usually mean that your microservice-based system becomes more mature. And it looks like you are ready for contract testing.

I set up a sample project to explain a contract testing concept in this article and show how to implement it with pact.

I would like to discover:

• The difference between contract tests and functional tests;
• Why end-to-end tests can’t replace contract tests;
• How easy is the pact to adopt;

I will use words Consumer (service consumer) and Provider (service provider) a lot because contract testing is based on testing a service from a perspective of its consumer.

I omit intentionally most of the implementation details in this article to focus on a general idea of contract testing with pact. However, in case you are interested in implementation of a use case you can find a description and a code base in this Github repo.

Consumer Part

Consumer drives and initiates the contract testing.

In my example consumer creates the contract tests for http API of a sample application using unittest, pact-python and pact-mock-service.

Pact mock service plays important role during the test run. It verifies if the requests from the tests were triggered and creates a pact (the instructions for Provider in JSON format).

These are the two test cases I implemented as contract tests:

1. Translation for "1"
Given Translation for number 1
When  I request a translation for 1
Then  I get response with expected body
And   The status code is 200
2. Translation for "-1"
Given No translation for number -1
When  I request a translation for number -1
Then  I get a response
And   The status code is 404

After I successfully execute the tests I have a pact generated for the next step — provider verification. This picture shows a workflow of the contract testing:

Provider Part

On this picture you can see pact generated by a mock service after the test execution:

This pact is complete but not ready. The provider states have to be implemented (e.g., “translation for number 1”).

They can be implemented within the service provider project or within a independent project — that’s for Provider to decide. The main idea is that the states setup should be possible to trigger through HTTP API by accepting a POST request from pact-verifier (a utility that executes instructions from a pact and installed together with pact-python). I wrote here (Github) better explanation about this part.

When the states are implemented Consumer or Provider can verify the instruction against a real service. This picture contains a command to run the verification and the execution result:

Impression

After reading about contract testing and working with pact I am summarising the main findings.

Contract testing does not focus on verification of the business logic of the service provider. In contrast to functional testing, contract testing verifies that a new version of the service doesn’t break the functionality of the service consumer and as a result a whole system functionality.

The contract tests are driven by Consumers. Provider can have many contracts with different Consumers.

It also becomes evident that contract testing requires a Consumer team to have a good communication with a Provider team.

I think contract testing is suitable for teams that work well together and already have certain test automation strategy in place and are looking for improvements.

About the pact. The tool was originally written in Ruby to my understanding and became a basement for implementation in different languages. From the first sight it seams intuitive to understand.

To setup my own project in Python I looked at this example for the pact-python repo. Because Python implementation incorporates Ruby parts it’s not always easy to track certain functionality of the tool.

The next step and a challenge could be to manage pacts and an infrastructure around them.

UPD. Here is a youtube video with a talk by Alon Pe’er on Move Fast and Consumer Driven Contract Test Things and slides from the talk are here.