CONTRACT TESTING FOR MICROSERVICES USING SWAGGER, PRISM AND DREDD
In the world of microservices or service oriented architectures, the contract is an agreement between the service provider and their consumers. Through this contract, each consumer knows how to use the service, what data can be sent and the responses it might receive. It provides the integrity needed for the system to remain stable.
Everything is fine until this point, problems may arise when the service needs to evolve and its API Specification changes.
For everything to work fine it’s necessary that the service implementation and their consumers fulfil the agreement. But, how we can be confident of that? It is at this point that contract testing is useful…
What is contract testing?
Basically, contract testing is a technique that is made up of writing tests which will ensure that both the server provider and, the consumer fulfil what the contract says.
Therefore, two perspectives exist:
Service provider side: Here is where we try to validate that the backend service carries out what is defined in the contract. If we talk about REST services, we should check for some points like:
- Is the service handling the request and response data properly? XML, JSON, query params…
- Is the data format accomplished? Maximum length, data types, required fields and so on…
- Are HTTP codes used correctly?
- Does the backend respond to the different request per resource / method?
Consumer side: Where we check whether the consumer is able to deal with the backend service as the contract describes or not. Therefore we should mainly check the consumer is ready to handle different input and output data types, response codes, etc… We mustn’t forget the fact that a consumer usually is a piece of software that will be injected in a higher level component (a web application, a mobile app, a batch process, a microservice etc…). If this small piece of software is not able to interact with the backend service properly the higher level component where it resides will fail during runtime and a bigger problem would emerge.
Why contract testing?
Because our systems are becoming more and more complex we don’t want them to fail for unknown reasons.
It’s quite common in microservice based systems that both the service provider and the service consumer teams are different. Usually, the first integration between the service and the consumer is accomplished without of the ordinary. Problems may show up when thinking of our microservices as immutable pieces of software that never change. Or even, when we think that a service implementation will never impact on the service contract (API Specification). Contract testing will help us to keep our contract in the middle of the integration process between the service provider and the consumer, and to anticipate when a change on the service contract will have an impact on it’s consumers:
- By a breaking change: the service client hosted in the consumer application will stop working properly with these contract changes.
- By backward compatibility: the consumers will be fine working with the new service contract. It’s up to the service consumer to choose whether to change it’s service client or not.
Usually, the contract could be the API Specification. This contract can be enriched by adding different scenarios for different consumers.
Contract testing from the service provider’s perspective
As we’ve already said, here we have to verify that the backend service accomplishes the API specification in order that the specification and its implementation don’t diverge. This is really important because breaking changes might happen and the integration between service and consumers will fail.
As we’ve already said, in service oriented architecture the service contract normally is the API specification. This can be enhanced by defining specific scenarios (given/when/then) using different kind of tools like Spring Cloud Contract or Pact.
It’s also strongly recommended to use a standard format to document our APIs such as Swagger, API Blueprint or RAML.
Let’s start with an example (you can check the whole project here):
We have an API document called ApiV1.yaml written using Swagger, recently known as Open API Specification (https://swagger.io/specification/).
Here we describe the different capabilities that the service offers:
- Orders search
- Single order search
- New order creation
All these things are exposed in a REST style.
Additionally, we describe the data types exchanged between the service and its consumers:
- Order
- New order
- Collection of orders
In relation to each data type, we describe every property on it: the property’s format, whether it is required or not, etc…
Let’s suppose we also have our backend service implemented according to the API specification and running on a server.
There is an interesting tool called Dredd that would be helpful by giving us a first diagnosis as straightforward as effective.
Using Dredd we can verify that our backend service is ready to fulfill with the examples defined on the API document. Even better is the fact that we don’t have to do anything.
The process is so easy, we only need to install Dredd wherever we want (our computer, CI server…) using npm:
> npm install -g dredd
Suppose the service is running on localhost:8080 and the API document is served in http://localhost:8080/awesomesite/ApiV1.yaml, we will execute the following command:
> dredd http://localhost:8080/awesomesite/ApiV1.yaml http://localhost:8080
The result will be:
But, what is really happening?
Dredd has sent some HTTP request to the backend service (hosted on localhost:8080). This requests have been made using the example data defined on the API document by the x-example property. Dredd expects the service to return a 2xx response as it is described on the API Specification. Each scenario to verify would be: given an endopint, when an example request will be sent, then the service has to return a 2xx response and the appropriate headers / body. This is what Dredd calls automatic expectations.
Now, let’s change the order status property in our backend service but not on the API document. We are going to modify its format from an integer to a string.
If we run dredd again we will get something like this:
As a result of the change, there is a gap among the service and its specification which Dredd is able to find. Moreover, this change in the service will have a bad impact on the service consumers by breaking the integration.
Of course, Dredd allows us writing tests for any other scenarios but to do that we have to write them by ourselves (hooks).
Contract testing from the service consumer perspective
At this point we aim to verify if the service consumer will be able to interact with the backend service through its API. We have to check if it will send request and handle responses correctly.
As we did in the previous section, we will start from the API specification. Here is where Prism comes into play (https://stoplight.io/platform/prism/).
Prism is able to fulfil some different actions but we are going to use it to create and run a fake implementation of our service API.
Once you have Prism installed and supposing src/main/webapp/ApiV1.yaml is the relative path to your API specification file, you have to execute this command:
> prism run --mock --list --spec src/main/webapp/ApiV1.yaml
By performing this action we will have a fake service instance running in http://localhost:4010 and ready to work. It will handle incoming requests, validate them and return responses according to the API Specification. Prism will make these responses on the fly.
One more time we have a way to verify whether our consumer would be able to interact properly with the service without having even one line of code in the backend service.
I personally love Prism because I find it so helpful when working with different teams. It allows the service consumer team starts to work only having the API specification.
Conclusions
By this post we wanted to review what contract testing is but not digging deeply in certain patterns like consumer-contract-testing (a very interesting topic by the way…).
There are some tools like Pact or Spring Cloud Contract that allow us to define specific scenarios (pacts) among consumers and services. However, a problem may show up due to an overlap between these pacts and the service specification (API document).
So, tools like Dredd and Prism give us a first diagnosis about the integration between service providers and consumers in a very simple way by just using the API specification.
It’s quite important to keep in mind that this technique isn’t an alternative to unit testing but a complement.
Many thanks to Daniel B. and Richard B. for helping me with this article.
