How to test Microservices with Consumer-Driven Contracts?

Mateusz Sokola
Dec 17, 2018 · 8 min read

Many software companies decide to build their applications in the distributed way. This approach gives high availability, and in the long run increases the development speed. The most common architectural pattern to design distributed systems are Microservices. The Microservice architecture is about splitting application into small standalone services, each of which plays a separate role in the overall system.

Imagine your client wants to build a job bulletin. The product must allow users to place job offers, and charge them a small listing fee. It also has to send several kind of e-mail notifications. Actually, it isn’t a single application, but rather a webpage, and iPhone and Android app. The first thing that needs to be done is to decide what architecture we will choose.

Benefits of Monolithic architecture

Pros:

  • Single codebase — simple to develop (at least in the beginning).
  • No need to verify integration with other services since they don’t exist.
  • Relatively easy deployment process — only one service needs to be deployed.

Cons:

  • Single point of failure — if one app is down, the whole service is down.
  • Maintenance complexity grows with time.
  • Hard to adapt new technologies and techniques. Usually it requires rewriting the application from scratch.

Benefits of Microservices

  • No single point of failure. When one service is down users can still use the application. Other services are still working.
  • Individual Microservices can be scaled up to increase their capacity and availability.
  • Security — most of the services aren’t exposed to the Internet.
  • Every service can be deployed independently of other services. Providing they don’t break the API contracts.

Cons:

  • Lots of work need to be invested to setup the foundation.
  • Deployment processes are more complex. Some deployments may require deploying more than one service.
  • Tons of work on DevOps. Especially with deployments…
  • Dealing with integration testing.

Most large applications are developed by hundreds of developers. That’s why splitting an application into smaller services makes work easier and faster. The Microservices architecture is a good choice for medium and large applications, although the most successful startups began their applications in monolithic architecture, and migrated to Microservices later on.

Designing Microservices for the Job Posting application

Imagine a user creates a new account. They enter the website, fill in the form, and send the request to the API Gateway. The API Gateway delivers this request to the User Service that creates a new account. After creating an account, User Service sends a request to the E-Mail Service to send an activation e-mail. An e-mail is then sent with a link that the user needs to click to activate their account.

As you noticed, the E-mail service isn’t available over the API Gateway. This is because only internal services can trigger e-mail notifications.

How to deal with testing?

Despite the tons of hardware needed to spin up the whole application stack, consider a testing hell. Let’s assume that budget isn’t a problem for your client. You have the necessary hardware, and a few DevOps guys to build and maintain the integration environment. Running all the tests takes so much time…

…Sometimes hours, seriously…

Some of our services look run cron and batch jobs. You can imagine it is nearly impossible to predict the time needed to execute them, meaning tests that rely on those jobs might be flaky.

What happens when your test relies on data processed by cron job, and it will try to confirm the test before the cron job has been finished? Your test fails, and you need to re-run your tests again…

…after a few attempts you get your build green. You wasted 3 hours on it, but now you can merge your changes, and deliver a feature…

Yay! Life is good!

…Unfortunately someone else merged something down two minutes ago, and so you need to merge those changes and run all the tests again…

Consumer-Driven Contracts

Example code can be found on Github.

How it works?

What tools to use?

You might also want to consider Dredd.

What is a consumer?

What is a provider?

If provider doesn’t meet the expectations they fail…

Hold on a second… so, how to deliver contracts to all peers?

  • a Git repository for storing pacts, and including them into each project with a git submodule. In my opinion the best way of doing it,
  • a file system.
  • the Pact Broker.

The Pact Broker is an application for sharing for consumer driven contracts and verification results. We can push our contracts there and allow the service providers to download them, and run tests against them.

DiUS provides a cloud version of the Pact Broker so, maybe you can take a look and use it. I don’t see too many advantages in running your own Pact Broker. In the end it is yet another service which requires maintenance from DevOps. If you want to run by yourself here are the Docker images.

I think storing contracts in a separate git repository is good enough. Personally i would create them during the integration pipeline, and run tests on the service providers within the same integration job— example integration pipeline here.

Why not Swagger?

It doesn’t mean you shouldn’t use Swagger at all, you can, and it is highly recommended. If you work with Microservics it is very likely that they are developed by many teams, and used by other teams (sometimes, external teams). Having documentation linked to your APIs makes development faster, because developers don’t need to figure out how to use endpoints. They can read it and try out on the documentation page.

You might want to read more about Swagger Mock Validator; a tool which allows you to verify contracts against the Swagger files.

Summary

If you are interested in Consumer-Driven Contracts I highly recommend watching the talk given by Mauri Edo and Ben Sayers from Atlassian.

I decided to keep all the code on my Github, so the article is easier to read.

If you have any problems or suggestions, please write a comment.

You can reach me on Medium, and Twitter.

HackerNoon.com

#BlackLivesMatter

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Mateusz Sokola

Written by

Senior JavaScript Developer based in Abu Dhabi, United Arab Emirates. https://www.youtube.com/channel/UCJV16_5c4A0amyBZSI4yP6A

HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean