Testing Pyramid vs Ice-Cream Cone

With the growth of application increases value of tests. It’s really important to check old parts (especially crucial business logic) of your application after implementing new features . So let’s consider some approaches.

There are 3 common types of tests: end-to-end, integration, unit.

The first one is end-to-end (or e2e) tests that offer to write tests from user perspective and focus on real world scenarios or user behavior. It’s really helpful, because you can establish how your bug would affect the user. It’s seems good but what can go wrong?

Very often one big bug can hide a number of small bugs, therefore it may be painfully to find-out where is the root cause of the problem (also test may cover huge peace of code — it doesn’t help us in this case). Bug can be anywhere in the code. Moreover developers should spend significant period of time to write and maintain tests. In addition these tests are slow and developers have to wait a lot until they will be confident that bug was fixed. That is a waste of time and money.

The next one is integration tests. It’s lite version of e2e tests, which only test interaction between small group of components/units, usually between two components. That means integration tests verify their behavior as a whole, that they work properly together. You may think smaller, write less and focus on possible bottlenecks.

But let’s think a little bit smaller. Unit tests can help us. They take only one component (unit) and test its in isolation. We may test their functions/API/state but only for one component. As a result our tests are really small, fast and isolated. Unfortunately we can’t say how test failure impacts on the user, but we can definitely say, which line of code is cause of the test failures. And it’s cool because existing of bug is not important for user. Resolved issue is more important.

Even if we covered each line of code by unit tests, we wouldn’t verify business logic as a whole. That means that we have to write each type of test to ensure that application is fully functional. If we didn’t find the bug, users would.

However to ensure efficiency and don’t waste of time it would be better to fill application with tests progressively. It’s better to start with unit tests and write them as much as possible (within reasonable limits). After that it’s a good idea to resolve omissions with integration tests and check how our linked components work together. After that to verify critical path of our application and to check it as a whole we can write a few number of e2e tests.

As we can see we have a lot of unit tests, fewer integration tests and small piece of e2e tests. Therein lies the Testing Pyramid. There is suggestion from Google’s developers to split their count in 70/20/10 percents. It’s reasonable, but these numbers may be different for your team and each project. This approach demonstrates good result and stability during development.

But there is anti-pattern which is named ‘Ice-Cream Cone’. The main idea is to reverse traditional testing pyramid (to write a huge amount of e2e tests, fewer integration tests and very little unit tests). So as we can see now it’s not a good idea, because we will spend a lot of time to write it, to maintain and to run it, and we will spend extra time to find out where is the cause of the bug.

So let me get this straight:

1) Testing pyramid provides approach where you can move to the next level after full filling of the previous level — it’s allows us to decrease cost of the test at all (unit tests are faster and cheaper than integration and, especially, e2e tests)

2) Also it better to use next-level tests to test that peaces of code, which can’t be tested by previous level of tests

3) Ice-cream cone is anti-pattern

4) Testing like ice-cream cone is slow, increasing build time

5) Unit tests can help us to find out root of the problem with integration or e2e test

6) But e2e tests are also important, because they can show how a failing test would impact the user

I’d like to say Thank You for reading article. If you liked it and if you want to learn more, please, let me know ;)
Have a good day!