The Test Pyramid
This article is a translation to English. The original version can be read here: https://medium.com/creditas-tech/a-pir%C3%A2mide-de-testes-a0faec465cc2.
The function of a test pyramid is to delineate the different levels of testing and give you a reference as to the number of tests there should be at each one of these levels.
At the top of the pyramid, we have end-to-end tests (or e2e). Their objective is to imitate the end user’s (whether it be a person, an API, or any other kind of client) behavior on an application.
At the pyramid’s base, we have unit tests, in which we verify the workings of our application’s smallest unit of testable code.
Between these two layers, we have integration tests. Its point is to verify if a set of units behave correctly, on a smaller scale than end-to-end tests.
End-to-end testing
These are tests that simulate a real environment, such as: opening the application in a browser, filling out forms, clicking buttons and eventually verifying if the outcome is what you expected. The difference between this kind of test and what an actual client does is that an end-to-end test usually happens in a controlled environment (instead of production) and the actions are executed by a bot (not a real user). It’s worth remembering that when we mention a “user”, we’re not necessarily talking about someone who will access your page or desktop application — if your software is an API, your user is the consumer of this API. (You can read more about the different types of end-to-end tests in this post by @katrina_tester.)
Unit tests
Unit tests verify the workings of an application’s smallest unit of testable code, independent of its interactions with other parts of the code.
This unit is generally seen as a public method or a class (when talking about Object-Oriented Programming), but it can also be seen as a set of classes/methods/objects interacting with each other. The discussion of what the size of each test should be is something I’ll leave to another article, but regardless of what your team normally uses, it should always be the smallest testable piece of your system.
What’s unique about unit tests is that, even though they’re focused on a small part of the system, they should be independent of the unit’s external collaborators.
In order for this to work, we need some “fake” objects which imitate the behavior of the real object (in a way that’s fast and deterministic), which are to be used in place of real collaborators. In fact, it’s not even necessary to have the real collaborator implemented when you’re writing unit tests. Lucas Santos wrote a post (in portuguese) about tests that has a section dedicated entirely to test doubles.
With this, our unit test is independent of any real external agent and can be really small and simple, and therefore fast to execute. Another advantage we have with this is that when unit tests fail, you know exactly where the problem is — no need to look for it!
Another great thing about unit tests is that they can guide the design of your code, especially when employing Test-Driven Development (TDD). Since the test acts as the code’s first user, we write code that is simpler and more cohesive. You can read more about TDD in Creditas’ post Test-Driven Development is Unnecessary!
Integration Tests
Unit tests are really dandy, really simple, and really fast; yet they aren’t enough.
We might have separately tested two units that will be integrated with each other, using the test doubles mentioned above, and concluded that they both function as desired. Despite this, it’s possible that they don’t work together (google “two unit tests, no integration tests” on Google Images and you’ll see what I’m talking about).
The solution to this problem is integration tests, which test how units work when they’re brought together. Different from end-to-end testing, these are tests that test functionality rather than the entire system (which is what we see with endpoint tests, for example). This implies that integration tests are:
- More complicated (to write and maintain) and time-consuming than unit tests, since they test an entire functionality (often with data persistence);
- Much simpler (to write and maintain) and quicker than end-to-end tests, since they only test one functionality at a time, not the whole application.
Returning to the topic of the test pyramid, it’s important to always remember that its bottom is easier to write and quicker to execute, while its top is more difficult and time-consuming.
With this in mind, the pyramid shows us the importance of your code being mostly covered in unit tests, since they’re very fast and much simpler (to write and maintain).
The more complex and time-consuming tests (end-to-end) should have fewer tests (this way, deploys won’t take more than an hour just to run your tests).
Integration tests exist for the scenarios that can’t be covered by end-to-end tests, and for scenarios that unit tests have already covered well (particularly because redundant tests are unnecessary). Following this logic, we have fewer integration tests than unit tests and a great deal less of end-to-end tests than integration tests.
At Creditas, the number of tests one of our systems has compared to the average time that each set of tests takes to run has the following relationship:
That’s all, folks! Now you know what the test pyramid is and what each one of its layers means, and what you should have in mind with regards to it as you construct your test suite :D
Interested in working with us? We’re always looking for people passionate about technology to join our crew! You can check out our openings here.
