Reduce build time by running tests in parallel
If you’ve been working on large projects, you might have observed that the build time increases with the number of features. Initially, it might have taken only a couple of seconds, but slowly, it increased to long minutes. Due to this, we sometimes try to avoid adding new tests or running them locally while developing. If we don’t add new tests, it might introduce bugs in the future.
As part of this blog, I’ll focus on how to reduce build time for Java projects. But before we start looking for solutions, let’s try to look into the problem. When we add new code to the codebase, we also add multiple layers of tests (for example, unit tests, integration tests, and so on). When we have lots of tests, it’ll take longer to complete the tests. So, now that the easy solution comes to our mind, let’s run tests in parallel instead of sequential. Yes, it’ll solve the problem, but the question is, how can we run tests in parallel? 🤔
Running tests in parallel
with Gradle
If you are using gradle, we can run tests in parallel by adding following configuration —
// build.gradle
test {
useJUnitPlatform() {}
minHeapSize="128m"
maxHeapSize="1024m"
// Use half of available processor to run tests in parallel.
// The following maxParallelForks config is suggested by gradle itself, https://docs.gradle.org/current/userguide/performance.html#execute_tests_in_parallel
maxParallelForks = Runtime.runtime.availableProcessors().intdiv(2) ?: 1
reports.html.required = true
}With Junit 5
If you are using Junit 5, you also can run tests in parallel by simply adding the following config junit-platform.properties file
# junit-platform.properties
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.config.strategy=dynamic
junit.jupiter.execution.parallel.mode.default=same_thread
junit.jupiter.execution.parallel.mode.classes.default=concurrentWith Junit, you can configure the concurrency level according to your need. Check the following image to understand the effect of junit.jupiter.execution.parallel.mode.default and junit.jupiter.execution.parallel.mode.classes.default config.
You can read more about the Junit config on their documentation.
Personally I prefer to set
junit.jupiter.execution.parallel.mode.defaulttosame_threadandjunit.jupiter.execution.parallel.mode.classes.defaulttoconcurrent. Even though it might take more time than running all tests concurrently (when both are set toconcurrent), it easy to maintain in long run.
With the above configuration, the tests started being executed in parallel, which works well for unit tests. But it might not work for integration tests, especially for Spring Boot integration tests annotated with @DirtiesContext. Currently, Junit doesn’t support parallel execution for integration tests. So, if we want to use Junit for parallel execution of integration tests, we need to make some changes.
Tweaks to use Junit to execute integration tests in parallel
Currently, Junit doesn’t support parallel execution as it’s tries to reuse context cache and there is no way to disable it. So, to simulate @DirtiesContext effect we need to create a new context cache key for each test class. The simplest way to generate a context key is to activate new random profile for each test class. Now the question is how can we activate a random profile? 🤔
We can do by using @ActiveProfiles annotation but we need to pass a custom resolver, something like following —
// CustomActiveProfilesResolver.java
package com.suman;
import java.util.Arrays;
import java.util.stream.Stream;
import org.springframework.test.context.support.DefaultActiveProfilesResolver;
import org.apache.commons.lang3.RandomStringUtils;
public class CustomActiveProfilesResolver extends DefaultActiveProfilesResolver {
@Override
public String[] resolve(Class<?> testClass) {
return Stream.concat(
Stream.of(testclass.getName().hashCode()),
Arrays.stream(super.resolve(testClass)))
.toArray(String[]::new);
}
}You can use random string instead of
hashCode(), but random string might not work for Nested test classes depends on project structure.
So if we put everything together then it’ll look something like following —
// HelloWorldIntegrationTest.java
package com.suman.hello;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.ActiveProfiles;
import com.suman.CustomActiveProfilesResolver;
@DirtiesContext
@ActiveProfiles(value = "test", resolver = CustomActiveProfilesResolver.class)
@SpringBootTest(webEnvironment = RANDOM_PORT)
class HelloWorldIntegrationTest {
@Test
void shouldAbc(){
// ...
}
@Test
void shouldAbcd(){
// ...
}
}Now with the above tweaks we can run our integration tests in parallel.
Running tests with common infra/data dependency
With all the above configurations, we were able to execute all tests (including integration tests) in parallel, but it creates issues once we have a dependency on some infrastructure or data.
For example, if we have database dependencies or are using tools to mock cloud infrastructure dependencies, we want the dependencies to be clean for each test; otherwise, we’ll face inconsistencies with the test results.
Here is the simplest option to have separate dependency instances, like a separate database and/or separate cloud infrastructure, for each class. So, now the question is how can we create separate cloud infrastructure or databases? 🤔
Here also, we’ll use our previous idea; we’ll create a random string and use the same random string as a prefix/suffix for each resource related to a specific test class. To achieve this automatically, we need to make a couple of changes —
- We will make use of the properties file (ex: application.properties) to pass all the names for common resources.
- Once we have moved all the hardcoded names from the Java file to the properties file, we will use the
ApplicationContextInitializerto update the values for the required properties. We can use the@ContextConfigurationannotation to provide the custom context initializer.
So, if we put everything together, our new tweaks will look something like the following —
// ConfigRandomizer.java
package com.suman;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.test.util.TestPropertyValues;
import org.springframework.context.ApplicationContextInitializer;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.core.env.ConfigurableEnvironment;
import org.testcontainers.shaded.org.apache.commons.lang3.RandomStringUtils;
import java.util.AbstractMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@Slf4j
public class ConfigRandomizer
implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final List<String> CONFIG_TO_BE_RANDOMIZED =
List.of("app.table.name", "app.config.random.prefix");
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
final ConfigurableEnvironment environment = applicationContext.getEnvironment();
final String prefix = RandomStringUtils.randomAlphabetic(5).toLowerCase();
log.info("===================== Using {} as prefix", prefix);
final Map<String, String> updateConfig =
CONFIG_TO_BE_RANDOMIZED.stream()
.map(
propertyName ->
new AbstractMap.SimpleImmutableEntry<>(
propertyName, "%s_%s".formatted(prefix, environment.getProperty(propertyName, ""))))
.collect(
Collectors.toMap(
AbstractMap.SimpleImmutableEntry::getKey,
AbstractMap.SimpleImmutableEntry::getValue));
TestPropertyValues.of(updateConfig).applyTo(applicationContext.getEnvironment());
}
}// HelloWorldIntegrationTest.java
package com.suman.hello;
import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.annotation.DirtiesContext;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.ContextConfiguration;
import com.suman.CustomActiveProfilesResolver;
import com.suman.ConfigRandomizer;
@DirtiesContext
@ActiveProfiles(value = "test", resolver = CustomActiveProfilesResolver.class)
@ContextConfiguration(initializers = ConfigRandomizer.class)
@SpringBootTest(webEnvironment = RANDOM_PORT)
class HelloWorldIntegrationTest {
@BeforeEach
void setUp() {
// ... create resources if needed
}
@AfterEach
void cleanUp() {
// ... cleanup resources if needed
}
@Test
void shouldAbc(){
// ...
}
@Test
void shouldAbcd(){
// ...
}
}Now with all these changes, our tests should work consistently, and we’ll observe that overall build time is reduced.
In case you need to find available port for tests, you can make use of
org.springframework.test.util.TestSocketUtils.findAvailableTcpPort()and update value inApplicationContextInitializer.
That’s pretty much what’s needed to run unit and integration tests in parallel.
Before I close, one thing I want to highlight is that by executing tests in parallel, we’ll be able to reduce the build execution time, but one thing we need to keep in mind is that parallelization is not a silver bullet. When we observe tests taking longer, we should go back to the code base and make sure we are following the proper TestPyramid concept. Only with the combination of TestPyramid and parallelization will we’ll be able to achieve the desired outcome.
I hope these concepts add some extra ammo to your arsenal. Feel free to provide your feedback and opinions 😄.
