Microservices — Contract Testing (BDCT) Part 1
Note: This article is written by assuming that the reader has a good understanding of Java programming language.
Microservice architecture is one of the most popular development approaches. Microservice architecture allows to separate a large application into smaller applications where each application is designed, developed, tested and deployed and we often call them as services. Services are built for business capabilities and each service performs a single function. Because they are independently run, each service can be updated, deployed, and scaled to meet demand for specific functions of an application. To serve a single user request, a microservices-based application can call on many internal microservices to compose its response. One of the benefits of this style of development is the faster delivery to the market.
Most of the times, a microservice can call another microservice to serve the user request. A microservice that sends the request is often called as the "Consumer" where as the microservice that responds to the request is often called as the "Provider". Both the Producer and Consumer have a common contract to communicate with each other. One of the biggest challenge is "communicating" the contract changes from provider to consumer. If the provider changes the contract without communicating to the consumer and deployed, it may break consumer service. This is very very serious problem and if the effected service is one of the critical services, it may lead to downtime and often lead to a massive revenue loss.
Imagine a scenario where a user wants to pay on e-commerce website by various payment methods like Credit Card, Bank Transfer, Other payment methods ( Google pay or Paypal or something else). Assume, in order to pay by Credit card, user has to save credit card details before proceeding with the payment. Lets assume this Save Credit Card Service will invoke a third party service which verifies the fraud activity on user's credit card.
- Front end service sends the request to Save Credit Card Service.
- Save Credit Card Service sends request to the external service Fraud Score Provider Service before saving Credit Card details into Database.
Now there is a dependency between Save Credit Card Service and Fraud Score Provider Service and imagine, what if Fraud Score Provider Service changes its contract and deploys these changes without communicating to Save Credit Card Service team??? I don't dare do it imagine the consequences. So we need to find a way to verify that provider changes does not introduce any breaking changes in the consumer service, before provider deploys its service into an Integrated Environment and we need to verify this in an isolated environment (CI) or in developers local machine. This is clearly a shift-left testing approach where testing will be carried as early as possible in the development life cycle which will help us to get the feedback faster.
In order to solve this problem, there are few testing processes available. One of them is Contract Testing.
Contract Testing: Contract testing is a methodology for ensuring that two separate systems (such as two microservices) are compatible and can communicate with one other. It captures the interactions that are exchanged between each service, storing them in a contract, which then can be used to verify that both parties adhere to it. There are two types of contract testing methods available.
- Consumer Driven Contract Testing
- Bi-Directional Contract Testing
Consumer Driven Contract Testing: As the name indicates, Consumer drives the contract testing. Consumer generates the contract with it's expectations (from the provider) and these expectations will be verified against the Provider.
Bi-Directional Contract Testing: Both the consumer and the provider publish their contracts to a tool, which verifies the consumer contract against the provider contract and tells us whether they are compatible or not.
How to generate Consumer Contract? : As a consumer, we can simply write down the consumer contract in a file and publish it. But, do you think its the right way to do it? what if we make any mistake while writing them?
So, instead of writing the consumer contract by hand, we can make use of some tools or libraries which can help us to generate the consumer contract. There are a few such tools available in the market like Pact Mock Server or Wiremock. Browse here for more tools available in the market.
We will write tests using Pact Mock Server or Wiremock where we verify the provider's behaviour and once all these tests will be executed, these tools will generate the consumer pact file. So here we are mocking the expected provider's behaviour, test it and generate the contract out of these tests.
In this article, we will focus on Pact DSL along with Pact Mock Server to generate consumer contract and also we will follow BDCT approach to verify the compatibility between the Provider's contract and the Consumer's contract. We use Pactflow from SmartBear to carry out the BDCT.
Steps to be followed in BiDirectional Contract Testing:
Provider's Steps:
- We will write provider contract (Open Api Specification) by hand or can be generated by code like swagger-generator etc.
- Once the Provider Contract is prepared, we need to make sure that this contract is working as expected before we publish to Pactflow. In order to verify this, we will make use of Provider's Integration tests and there are some tools available to verify the provider's contract. Some of those tools are RestAssured in conjunction with Atlassian's OpenApiValidator, Postman, Dredd etc.
- Once the Provider's Contract is verified in local, we will publish it to Pactflow along with verification results.
- We will run "can_i_deploy" command to check if we can deploy the provider service.
- We will record the deployment in Pactflow once we deploy our provider service.
Consumer's Steps:
- We will mock the Provider's behaviour using Pact DSL along with Pact Mock Server and will write tests to verify this behaviour. Once these tests will be executed, a Pact file (Consumer Contract) will be generated at the end of the execution.
- We will publish the Consumer contract to Pactflow
- We will run “can_i_deploy” command to check if we can deploy the consumer service.
- We will record the deployment in Pactflow once we deploy our consumer service.
Lets Code:
Pre-requisites:
- Create an account in https://pactflow.io/ and copy read/write token from Pactflow settings page.
2. Set these tokens as environment variables in your machine
export PACT_BROKER_BASE_URL="https://<your>.pactflow.io"
export PACT_BROKER_TOKEN="Token"NOTE: Above information has a bit of sensitive data, general practice is to store this kind of sensitive information in a secured place like Vault and retrieve them in your CI or you can also save them in your CI configuration as protected variables.
3. Clone https://github.com/AnjiB/SimpleApiTest and build it.
4. Clone provider service: https://github.com/AnjiB/ccfc and build it.
Note: If you want to see the APIs exposed by "Fraud Score Provider Service", start the application by running the command "mvn spring-boot:run" and launch: http://localhost:8080/v3/api-docs.
5. Clone consumer service: https://github.com/AnjiB/save-cc and build it.
Note: If you want to see the APIs exposed by “Save Credit Card Service”, start the application by running the command “mvn spring-boot:run” and launch: http://localhost:8081/v3/api-docs.
6. Start Docker in your local.
7. Cd to ccfc and execute provider verification test.
mvn test
01:15:53.684 [SpringApplicationShutdownHook] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Shutdown initiated...
01:15:53.686 [SpringApplicationShutdownHook] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Shutdown completed.
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------8. Let's test if OpenApiValidator throws errors if we make any changes to the Provider contract. In order to test this scenarios, open ccfc_provider_contract.json and update the CCFraudInfo schema field "error" type from String to Integer.
"components": {
"schemas": {
"CCFraudInfo": {
"type": "object",
"properties": {
"timeStamp": {
"type": "string"
},
"status": {
"type": "integer",
"format": "int32"
},
"error": {
"type": "integer"
},
"path": {
"type": "string"
},
"ccNumber": {
"type": "string"
},
"issueDate": {
"type": "string",
"format": "date"
},
"expDate": {
"type": "string",
"format": "date"
},
"fraudProps": {
"$ref": "#/components/schemas/FraudProps"
}
}
},9. Save the change and execute provider verification test again and observe the failures. These failures tell us that in our provider contract we defined the "error" field type as Integer but our application is responding with String type. This is how we make sure that our application behaves as per the spec we defined.
[ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 8.968 s <<< FAILURE! - in com.anji.finance.ccfc.CcfcProviderVerificationTest
[ERROR] testErroMessageForInvalidCreditCard Time elapsed: 1.661 s <<< ERROR!
com.atlassian.oai.validator.restassured.OpenApiValidationFilter$OpenApiValidationException:
{
"messages" : [ {
"key" : "validation.response.body.schema.type",
"level" : "ERROR",
"message" : "[Path '/error'] Instance type (string) does not match any allowed primitive type (allowed: [\"integer\"])",
"context" : {
"requestPath" : "/v1/fraudcheck/score",
"responseStatus" : 500,
"location" : "RESPONSE",
"pointers" : {
"instance" : "/error",
"schema" : "/properties/error"
},
"requestMethod" : "GET"
}
} ]
}
at com.anji.finance.ccfc.CcfcProviderVerificationTest.testErroMessageForInvalidCreditCard(CcfcProviderVerificationTest.java:54)
[ERROR] testErroMessageForInvalidCreds Time elapsed: 0.058 s <<< ERROR!
com.atlassian.oai.validator.restassured.OpenApiValidationFilter$OpenApiValidationException:
{
"messages" : [ {
"key" : "validation.response.body.schema.type",
"level" : "ERROR",
"message" : "[Path '/error'] Instance type (string) does not match any allowed primitive type (allowed: [\"integer\"])",
"context" : {
"requestPath" : "/v1/fraudcheck/score",
"responseStatus" : 401,
"location" : "RESPONSE",
"pointers" : {
"instance" : "/error",
"schema" : "/properties/error"
},
"requestMethod" : "GET"
}
} ]
}
at com.anji.finance.ccfc.CcfcProviderVerificationTest.testErroMessageForInvalidCreds(CcfcProviderVerificationTest.java:76)
01:21:04.891 [SpringApplicationShutdownHook] INFO o.s.o.j.LocalContainerEntityManagerFactoryBean - Closing JPA EntityManagerFactory for persistence unit 'default'
01:21:04.891 [SpringApplicationShutdownHook] INFO o.h.t.s.i.SchemaDropperImpl$DelayedDropActionImpl - HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
Hibernate: drop table if exists CcFraudScoreInfo CASCADE
01:21:04.893 [SpringApplicationShutdownHook] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 90121, SQLState: 90121
01:21:04.893 [SpringApplicationShutdownHook] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - Database is already closed (to disable automatic closing at VM shutdown, add ";DB_CLOSE_ON_EXIT=FALSE" to the db URL) [90121-214]
01:21:04.894 [SpringApplicationShutdownHook] WARN o.h.e.jdbc.spi.SqlExceptionHelper - SQL Error: 90121, SQLState: 90121
01:21:04.894 [SpringApplicationShutdownHook] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - Database is already closed (to disable automatic closing at VM shutdown, add ";DB_CLOSE_ON_EXIT=FALSE" to the db URL) [90121-214]
01:21:04.895 [SpringApplicationShutdownHook] WARN o.s.b.f.s.DisposableBeanAdapter - Invocation of destroy method failed on bean with name 'entityManagerFactory': org.hibernate.exception.JDBCConnectionException: Unable to release JDBC Connection used for DDL execution
01:21:04.895 [SpringApplicationShutdownHook] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Shutdown initiated...
01:21:04.897 [SpringApplicationShutdownHook] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Shutdown completed.
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] CcfcProviderVerificationTest.testErroMessageForInvalidCreditCard:54 » OpenApiValidation
[ERROR] CcfcProviderVerificationTest.testErroMessageForInvalidCreds:76 » OpenApiValidation
[INFO]
[ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------10. Now the revert the changes in spec file, run the test again and we are good to publish our provider contract to Pactflow.
11. Execute below shell script to publish the provider contract. Alternatively you can run this shell script provided in scripts folder.
docker run --rm -v /${PWD}:/${PWD} -w ${PWD} \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
pactflow publish-provider-contract docs/oas_contract/ccfc_provider_contract.json \
--provider fraud-service-provider \
--provider-app-version 1.0.0\
--branch main \
--content-type application/json \
--verification-exit-code=0 \
--verification-results target/surefire-reports/TEST-com.anji.finance.ccfc.CcfcProviderVerificationTest.xml \
--verification-results-content-type text/xml \
--verifier restassured
12. Login to your Pactflow account and observe that we publish our provider contract successfully.
13. Run can-i-deploy shell script which says Yes as there is no consumer contract at this moment.
14. Assume we deployed this service to production and now we need to record this deployment in Pactflow.
15. Now it's the time to publish the Consumer contract. Cd to "save-cc" folder and run the command — "mvn test" and it will create the pact file in "target/pacts/" folder.
{
"provider": {
"name": "fraud-service-provider"
},
"consumer": {
"name": "save-credit-card-consumer"
},
"interactions": [
{
"description": "Invalid Auth Error message from ccfc service",
"request": {
"method": "GET",
"path": "/v1/fraudcheck/score",
"headers": {
"Authorization": "Basic afaaf="
},
"query": {
"cc": [
"4532788397355156"
],
"exp": [
"2023-03-31"
]
}
},
"response": {
"status": 401,
"body": {
"error": "Bad credentials"
}
},
"providerStates": [
{
"name": "Invalid Auth Token is Provided"
}
]
},
{
"description": "Error message from ccfc service",
"request": {
"method": "GET",
"path": "/v1/fraudcheck/score",
"headers": {
"Authorization": "Basic afafaf="
},
"query": {
"cc": [
"test"
],
"exp": [
"2023-03-31"
]
}
},
"response": {
"status": 500,
"body": {
"error": "There is no credit with given card number. Please enter valid credit card"
}
},
"providerStates": [
{
"name": "Invalid credit card is provided"
}
]
},
{
"description": "Invalid CC Error message from ccfc service",
"request": {
"method": "GET",
"path": "/v1/fraudcheck/score",
"headers": {
"Authorization": "Basic afaffa="
},
"query": {
"cc": [
"test"
],
"exp": [
"2023-03-31"
]
}
},
"response": {
"status": 500,
"body": {
"error": "There is no credit with given card number. Please enter valid credit card"
}
},
"providerStates": [
{
"name": "Invalid credit card is provided"
}
]
},
{
"description": "Fraud score for a valid credit card",
"request": {
"method": "GET",
"path": "/v1/fraudcheck/score",
"headers": {
"Authorization": "Basic afaff="
},
"query": {
"cc": [
"4532788397355156"
],
"exp": [
"2023-03-31"
]
}
},
"response": {
"status": 200,
"body": {
"ccNumber": "4532788397355156",
"issueDate": "2019-03-31",
"expDate": "2023-03-31",
"fraudProps": {
"issuer": "VISA",
"fraudScore": 700
}
}
},
"providerStates": [
{
"name": "Valid credit card is provided"
}
]
}
],
"metadata": {
"pactSpecification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "4.1.7"
}
}
}16. Run the shell script — "publish_consumer_contract" to publish the consumer contract.
17. Log in to Pactflow account to check the status
18. Run can-i-deploy script to check if we can deploy consumer application
19. Deploy the consumer application and record this deployment in Pactflow.
Now we have consumer with version 1.0.0 which is compatible with provider with version 1.0.0
20. Lets introduce a breaking change from the provider and see if we can deploy it or not. In order to carry this test, update field "error" type from "String to Integer" in spec file and publish it as version 2.0.0
Note: Your Provider Verification Test will fail with above change and you need to fix these errors by changing the application behavior.
docker run --rm -v /${PWD}:/${PWD} -w ${PWD} \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
pactflow publish-provider-contract docs/oas_contract/ccfc_provider_contract.json \
--provider fraud-service-provider \
--provider-app-version 2.0.0\
--branch main \
--content-type application/json \
--verification-exit-code=0 \
--verification-results target/surefire-reports/TEST-com.anji.finance.ccfc.CcfcProviderVerificationTest.xml \
--verification-results-content-type text/xml \
--verifier restassured21. Login to Pactflow and check the compatibility
22. Since we are introducing the breaking change, can we ask the Pactflow if we can deploy this version to the production?
Let see what Pactflow says..
docker run --rm -v /${PWD}:/${PWD} -w ${PWD} \
-e PACT_BROKER_BASE_URL \
-e PACT_BROKER_TOKEN \
pactfoundation/pact-cli:latest \
pact-broker can-i-deploy \
--pacticipant fraud-service-provider \
--version 2.0.0 \
--to-environment production \
--retry-while-unknown 6 \
--retry-interval 10
Conclusion: Bi-Directional Contract Testing is one of the easiest ways to implement the Contract Testing ( From the Provider’s perspective ) and we can take advantage of the existing Provider’s Integration Tests to verify its Open API specification documentation before we share to all of it’s consumers.
References:
1. https://pactflow.io/
