Test Pyramid and Microservices
How to define the best suit test for your architecture?
Microservice is a software architecture pattern trendy, and it’s pretty simple to understand why. It gives us many advantages, such as code maintainability, scalability, technological flexibility, and the possibility of working as autonomous teams. However, there are disadvantages, like time degradation due to network latency or the complexity of the monitoring environment.
But one of the most challenging disadvantages is finding a good way to test our microservices. To solve this problem, let’s look at the visual metaphor Test Pyramid and understand how to use it to define the best suit test for your Microservice Architecture.
Tests are complex and challenging.
A few years ago, it was common to find teams composed of only QA engineers who received the software already done. This team was responsible for ensuring it matched the requirements to go live, and tests were usually manual. This approach has two big problems:
- Manual tests are slow. You should not tell a human to do a task that a machine can do much faster and 24 hours x 7 days.
- The tests were executed at the end of the development process. The fact that the QA engineer received the software after the development was finished, and didn’t participate in the creation of test scenarios, led us to rework and have a hard time understanding and completing the work. Furthermore, this process creates some friction between development and QA teams.
Besides that, add a bit of complexity when choosing Microservices Architecture, spreading responsibilities between different services and wanting to create a test scenario involving more than one service.
In the good old times of Monolith Architecture, we could probably solve this problem by calling methods in different classes/modules of our software. It doesn’t mean monolith doesn’t have external dependencies; they have. There is always a database connection or a file system, but microservices have much more dependencies since we have multiple services communicating between them to solve problems in different domains.
To help us to reduce this complexity and identify the best approach to test and ensure the quality of our software, we can use the visual metaphor of the Test Pyramid.
The central concept here is that there are different tests: unit, integration, component, and end-to-end. These types of tests are in different layers of a pyramid.
The tests further up the pyramid are more complex, slower and more expensive to write and execute. Tests further down the pyramid are less complex, faster and more reliable to write and execute.
So should we focus on writing only faster, less complex and most reliable tests, right?
Well, actually, no. It’s not that simple.
At the bottom of the pyramid, we find the Unit Tests. They are faster and more complex because their responsibility is to test only class and isolated methods. At the top of the pyramid, we find the End-to-end Tests. They are costly and unreliable because their responsibility is to test flows that can involve hundreds of different services and external dependencies and present many scenarios we need to cover. Both types of tests have their value, but knowing their differences is the key to helping us decide where we must put more effort regarding the system we want to validate.
According to Chris Richardson, author of Microservices Patterns, the ratio we should invest in our tests is the same as we see in the Test Pyramid image.
The lower the level, the most tests we should have; the higher the level, the fewer tests we should have. Again, it doesn’t mean we shouldn’t have any tests of the most elevated type in the pyramid, but we should have more tests of the lowest type.
To understand better, let’s see more details about each test.
Unit Test
Unit Tests are responsible for validating the smallest testable piece of code. They should test a class method, and they must be quick.
Important to observe that a unit test should test a method’s public interface. The private methods are usually considered implementation details and should not be tested in an isolated way. If you find yourself in this situation often, it may signal that your class is too complex and probably needs a refactor. But don’t worry, this is good! Tests help us to evaluate our code design better.
Martin Fowler likes to divide unit tests into Sociable and Solitary. It is important to recognise when to use them.
Solitary Unit Test
Imagine you want to test a class with a method which calculates the price of some product. Let’s also imagine this method must execute functions from the Customer and Product classes. If we want to write a unit test of Solitary type, then we don’t want to use the real Customer and Product classes because a failure in these classes can create chaos and also fail our test. To solve this, we need to use Test Doubles.
Test Double is a generic term for when we want to replace objects and dependencies of the method we want to test, and some examples are Dummy, Fake, Stubs, Spies and Mocks. You probably are familiar with Mocks, but it is good to know that other options exist to solve different problems with unit tests.
Sociable Unit Test
But sometimes, you want to test your method’s behaviour given all the business rules in methods of other classes that you call, as long as it’s not an external dependency like API or database. In this situation, you must write a unit test of the Sociable type.
In summary, you should write unit tests. Write them a lot. They should be fast, and their responsibility is to test our code’s behaviour through public methods. You should always think about which type of unit test will be more effective in each situation. Choose wisely between Sociable or Solitary, depending on whether you want to focus on testing the state of objects and interactions or if you want to validate the behaviour through all calls that contain some business rule, even if it belongs to other classes.
Integration Test
Integration tests ensure that our service can interact with external dependencies, especially with other services. Here we’ll guarantee that the provider and consumer will respect the contract defined for integration between services.
To solve this problem, we could create a test to execute a REST call to an actual running service. For sure, this should work but remember the pyramid. If we choose this, we are creating end-to-end tests, which are brittle, costly, and hard to run. What should we do if the service we are testing is down? And what if the database is down? Which environment should we use, DEV or TEST? Can I add these tests to my CI/CD pipeline?
The integration test goal is to ensure that an API contract is correct and must be fast and reliable to run multiple times. Because of this, we don’t write integration tests using external DNS or real running applications. We must choose other approaches.
In his book Microservices Patterns, Chris Richardson brings three alternative patterns to Microservices Integration Tests.
For the future, let’s define that:
Consumer is the team interested in consuming a service
Provider is the team responsible for providing the service that the consumer team needs.
Consumer-Driven Test
The first pattern is called Consumer-Driven Test. Here the consumer team writes a contract test validating the contract agreed upon between the teams and then includes this test in the service repository that belongs to the provider team. This process can be done using a Merge Request or a copy and paste code. The test should be executed in every build, and then both teams ensure the provider team will never break the contract expected by the consumer. If the provider team breaks the contract, it could be identified quickly and before deployment.
The downside of this approach is that it could be a little invasive to have another team committing to your repository. It can increase the complexity of your merge request flow; imagine having a lot of different teams sending merge requests to you. What if some tests came with an error? What if the code is ugly, dirty and repeated? The key to making this approach work is keeping the team’s communication in good shape.
Provider-Driven Test
The second pattern is called Consumer-Side Contract Test, sometimes called Provider-Driven Contract Test. The idea here is pretty simple; the provider team is responsible for writing the contract tests of their endpoints. This approach is the most common and easiest to apply since the provider team knows the business rules, the business flow and the contracts they define together with the consumer team. The downside of this approach is that the provider team can choose not to write the contract tests or skip them to spare time. Another problem is if the provider team makes some changes in the contract and test but does not tell the consumer team about the changes, they create a bug in the service. If contract tests do not exist or do not reflect the reality of the contract expected by consumers, we can only identify problems in the LIVE environment.
Consumer-Driven Test (but with steroids)
The third pattern is to use frameworks focused on solving contract test problems. Two examples of these frameworks are Pact and Spring Cloud Contract. These frameworks can have minor differences, but the central concept it’s quite the same, a mutual collaboration of all parts. Spring Cloud Contract provides the possibility of writing contracts using Groovy. The flow proposed by Chris Richardson is:
- The consumer writes the contract expected and sends it to the provider team.
- The provider team uses the contract created to write the tests using Spring Cloud Contract.
- The provider team publishes the contract tested in a contract repository.
- The consumer reads the contracts from the repository to develop their tests in their repository.
OK, this can be a bit confusing, but the idea here is that the Contract Repository keeps updated, and all the teams involved in the process can ensure they are using and testing the correct version of the defined contract. With this, they also ensure that the dependencies will not be broken.
It’s good to remember that the examples above are related to REST API, but we can (and should) test our contracts of event integrations too. If event integration is an important part of your work, you can choose any of the three patterns presented before to test it.
In summary, integration tests are an essential part of software engineering and are often forgotten. To understand the importance of writing integration tests, we just need to reflect on how the biggest challenges we face are related to the integration between our software.
Integration is present in any project. Microservice Architecture added a layer of complexity to this topic, and because of this, we need to keep in mind the importance of integration tests.
Component Test
Component tests are responsible for ensuring the correctness of the service as a whole. The idea is to write tests to analyse a service’s complete business flow.
OK, now you think you must write end-to-end tests to achieve this level of ensuring, but this will bring other issues, like how to deal with external dependencies in a fast and reliable way. But the answer to this is you don’t deal with it; you choose another approach. You choose to write component tests.
The idea here is to replace any external dependency with Test Doubles. Then we can write tests that pass through all the processes of our service, all communications between our classes, and validate our domain.
To achieve this, first, we need to define our acceptance tests. In component tests, the use of Behavior Driven Development is a strong ally. We should identify scenarios and set the expected behaviours, replace all external dependencies like database and another service with Test Doubles, and finally check if your service’s functionality matches your expectations. Here is an example of a test scenario written using BDD:
Feature: Place OrderAs a consumer of the Order Service
I should be able to place an OrderScenario: Order authorized
Given a valid consumer
Given using a valid credit card
Given the restaurant is accepting orders
When I place an order for Chicken Vindaloo at Ajanta
Then the order should be APPROVED
And an OrderAuthorized event should be publishedScenario: Order rejected due to expired credit card
Given a valid consumer
Given using an expired credit card
Given the restaurant is accepting orders
When I place an order for Chicken Vindaloo ar Ajanta
Then the order should be REJECTED
And an OrderRejected event should be published
There are awesome tools to use BDD, like Cucumber, but the important thing here is to use business experts’ knowledge. One advantage of writing tests using BDD is to use a high-level language, which brings developers the most important thing in software development: business knowledge.
End-To-End
Last but not least, here are the end-to-end tests. Feared by developers and the favourite process of old-school QA engineers.
The idea here is pretty simple: I must make my software work for real.
If component tests can thoroughly test individual service processes, end-to-end tests can validate the correctness of multiple services with their dependencies in an integrated way.
Now there is no escape. We write a test that will simulate a regular day of business operation with the software we developed. We must ensure that every integration works for real; for this, we need every dependency running UP and READY to be used.
We can use the same strategy applied to the component tests to achieve the expected results: a BDD tool like Cucumber with well-defined and described test scenarios. The difference is, of course, now we have lots of scenarios to validate, and all services must be READY to answer the flow requests.
End-to-end tests are costly. We need to ensure that all involved infrastructure is ready and working correctly. We must ensure that all involved services are deployed with the correct version and running perfectly. We also need to ensure that all involved users have access to the test environment and data. Sometimes we also need to pray.
End-to-end tests are at the top of the pyramid. They are slow, costly, complex and non-reliable. They should not be executed with the same frequency as the other tests. The chances of failing are pretty big, and the chances of a false positive are even bigger. Because of this, end-to-end tests are the type that we should write less.
Conclusion
The pyramid test is a visual metaphor that helps us to understand how the tests are divided, by their complexity, cost and execution time.
With this knowledge in mind, we can plan our strategy better to develop the best for testing and ensure the quality of our software delivery.
All types of tests are important. They all have their value and should not be ignored. We should focus on the type of test that will bring us more value in our software development. It always depends on the situation!
The most important thing is to find the balance and keep the proportionality.
Always write more unit tests, followed by integration tests, component tests, and end-to-end tests. But remember: writing tests is a fundamental part of writing software.
Do you think you have what it takes to be one of us?
At WAES, we are always looking for the best developers and data engineers to help Dutch companies succeed. If you are interested in becoming a part of our team and moving to The Netherlands, look at our open positions here.
WAES publication
Our content creators constantly create new articles about software development, lifestyle, and WAES. So make sure to follow us on Medium to learn more.
Also, make sure to follow us on our social media:
LinkedIn — Instagram — Twitter — YouTube
References
Chris Richardson, Microservice Patterns: With examples in Java
https://microservices.io/
https://martinfowler.com/bliki/UnitTest.html
https://martinfowler.com/bliki/TestPyramid.html
https://martinfowler.com/articles/microservice-testing/
