How to optimize the runtime of your SpringBoot integration tests

When you start with a clean SpringBoot project the overall build time (“Gradle build” or “maven install”) is very fast. With increasing productive code (e.g. multi-module configuration) and different kinds of tests (e.g. unit tests, component tests, integration tests, e2e tests, …) also using separate source sets (e.g. to separate unit from integration tests), the build time is increasing silently second by second.

As long as the tests are mainly unit tests mocking external dependencies, the increase of build time is vanishingly small (~ 1–5 ms per test). But if the project has a lot of integration tests using application context and real database the slow down of the build time with every additional test is noticeable. The database startup and the application context's initialization have the largest effect on runtime. So what to do?

SpringBoot (org.spring framework.boot:spring-boot-starter-test) offers a number of different strategies to reduce test runtime:

• Test slices by annotations (e.g. MockMvcTest + specify a single rest controller to only load related beans - see https://spring.io/guides/gs/testing-web/ )
• ActiveProfiles annotation in combination with using test stubs instead of real implementation in the application context (see https://docs.spring.io/spring-framework/docs/current/reference/html/testing.html#spring-testing-annotation-activeprofiles)
• Test Context Configuration only loading specified beans to application context (see https://docs.spring.io/spring-framework/docs/current/reference/html/testing.html#spring-testing-annotation-contextconfiguration)
• …

This options can lead to reduced setup time of application configuration because of fewer beans to load and also using stubs, which have a faster execution time.

This seams to be helpful in first thought, but when having a look for the test runtime of integration test can see the following (even in my sample project with only a couple of beans):

The initial setup of every test class (setup of application context + start of database) takes a multiple of the time every following test takes for execution. This means the initial setup of the application context is the bottleneck.

Searching in the reference of spring test I could find the following section: https://docs.spring.io/spring-framework/docs/current/reference/html/testing.html#testcontext-ctx-management-caching

Spring provides a caching mechanism for application context when running tests. When running integration tests using application context, a cache is created using a map with the combination of configuration parameters as a unique key and the initialized application context as a value. The default size is set to 32 which means when the cache is full, a least recently used (LRU) policy is used to determine the removed entry and the application context is closed.

As you can see in the spring documentation the key of the cache is composed of different configuration parameters. To have a better overview of how a test configuration is affecting the cache, it is possible to activate logging for this particular functionality.

Add the below line to the application.properties:

Running the tests the following entries appear inn log:

2022–07–04 08:05:24.762 DEBUG 20808 - - [ Test worker] org.springframework.test.context.cache : Spring test ApplicationContext cache statistics: [DefaultContextCache@6d9bcf19 size = 3, maxSize = 32, parentContextCount = 0, hitCount = 142, missCount = 3]

In the above example, this means I have 3 integration test classes and after running the last of the tests there are 3 application contexts cached. Not very good…

To benefit from the caching mechanism the following topics are important:

Keep application state clean

Remove all resources created during test execution (e.g. entries in the database, files, …) and reset the internal state (e.g. stateful components).

Try to use complete application context for testing

Using different compositions of application context leads to every configuration creating a new entry in the context cache.

Not use different profiles for testing (@ActiveProfile annotation)

Every profile leads to a new entry in the context cache.

Not use @DirtiesContext annotation

Using this annotation leads to a drop in the application context after tests of one class are executed.

Only use @MockBean/@SpyBean if necessary

If some tests use mocks or spies a separate application context is used because the component scan has 2 different components of the same type.

As a result of the optimizations its possible to reduce the count of application context used for integration tests.

In my sample project I could reduce the used contexts to 1:

2022–07–04 08:18:04.265 DEBUG 10836 - - [ Test worker] org.springframework.test.context.cache : Spring test ApplicationContext cache statistics: [DefaultContextCache@373c367 size = 1, maxSize = 32, parentContextCount = 0, hitCount = 136, missCount = 1]

In real projects, it’s not possible to re-use one application context for all tests because especially the usage of @MockBean/@SpyBean is part of testing exceptional cases (like database exceptions) which cannot be reproduced easily using productive code.

But as a result of being aware of the caching mechanism of spring boot I could reduce the amount of used application context for the integration tests for one module of a real-time project with at the moment ~ 120 tests from 32+ (contexts are dropped during execution because exceeding the limit) to 9.