Maven Phase Testing COMBO — Cheat Sheet
The full cheat sheet source code is available on GitHub.
Small, seemingly insignificant configurations and settings of basic tools of our daily work. Usually, we don’t introduce them too often, so some people, including me, may easily forget them. With the help come cheat sheets, thanks to which we immediately find what we need.
This article contains the cheat sheet for optimal tests configuration and execution in Apache Maven.
Featuring with
- Enables multilevel testing (“TestPyramid”), where tests are split into two groups, low-level unit tests and high-level integration tests
- Applies and simultaneously handles JUnit 4, JUnit 5 and TestNG frameworks
- There is a possibility to execute tests, from one or both groups, in the failsafe mode
- There is a possibility to exclude tests, from one or both groups
- There is a possibility to narrow tests, from one or both groups, to WIP tests
Preparing Maven
Tests are split into two groups which are executed in separate Maven phases. An execution in separate plugins enables independent management of each test group.
- Unit tests in the “test” phase by Maven Surefire Plugin
- Integration tests in the “verify” phase by Maven Failesafe Plugin
<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
</plugin>
</plugins>
</build>
Multiple tests frameworks support
In order to detect and process distinct tests frameworks, dedicated frameworks providers need to be specified explicitly as plugins dependencies (the same dependencies in both plugins).
Additionally there is a need to exclude JUnit tests handling from TestNG provider, which by default supports JUnit as well.
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit-platform</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
</dependencies>
<configuration>
<properties>
<!-- disables JUnit tests for TestNG provider -->
<property>
<name>junit</name>
<value>false</value>
</property>
</properties>
</plugin>
Marking tests
Unit and integration tests are marked and separated on the basis of specific naming patterns. There is no additional configuration needed.
Unit tests names need to be compliant with the default Maven Surefire Plugin includes pattern.
Integration tests names need to be compliant with the default Maven Failesafe Plugin includes pattern.
Exemplary unit test
class ExampleTest {
..
}
Exemplary integration test
class ExampleIT {
..
}
Failsafe mode
Enabled through common plugins configuration property <testFailureIgnore>.
<properties>
<testFailureIgnore>false</testFailureIgnore>
<testFailureIgnoreITs>${testFailureIgnore}</testFailureIgnoreITs>
<testFailureIgnoreUTs>${testFailureIgnore}</testFailureIgnoreUTs>
</properties><plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<testFailureIgnore>${testFailureIgnoreITs}</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<testFailureIgnore>${testFailureIgnoreUTs}</testFailureIgnore>
</configuration>
</plugin>
Set through custom Java system properties
. By default executions are not failsafe.
- -DtestFailureIgnoreITs — ignores failures for integration tests
- -DtestFailureIgnoreUTs — ignores failures for unit tests
- -DtestFailureIgnore — ignores failures for unit and integration tests
For unit tests
$ mvn -DtestFailureIgnoreUTs ..
For integration tests
$ mvn -DtestFailureIgnoreITs ..
For unit and integration tests
$ mvn -DtestFailureIgnore ..
Exclusion
Enabled through plugins configuration properties.
<properties>
<skipITs>${skipTests}</skipITs>
<skipUTs>${skipTests}</skipUTs>
</properties><plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<skipTests>${skipUTs}</skipTests>
</testFailureIgnore>
</configuration>
</plugin>
Set through Java system properties
. By default there are no executions.
- -DskipITs — skips integration tests (Maven Failsafe Plugin property)
- -DskipUTs — skips unit tests (custom property)
- -DskipTests — skips unit and integration tests (Maven Surefire Plugin property)
Skips unit tests
$ mvn -DskipUTs ..
Skips integration tests
$ mvn -DskipITs ..
WIP narrowing
There is a need to configure plugins groups property and test needs to be marked by an annotation.
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<configuration>
<groups>${maven-failsafe-plugin.groups}</groups>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<groups>${maven-surefire-plugin.groups}</groups>
</configuration>
</plugin>
- JUnit 4 tests marked by the class and method level
@Category
annotation.
import org.junit.experimental.categories.Category;
import pl.edc.dc.UnitTest;@Category(WipTest.class)
public class JUnit4Test {
..
}
- JUnit 5 tests marked by the class and method level
@Tag
annotation.
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;@Tag("pl.edc.dc.WipTest")
class JUnit5Test {
..
}
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;@Test(groups = {"pl.edc.dc.WipTest"})
public class TngTest {
..
}
WIP narrowing is activated through a dedicated Maven build profile
for the given testing phase.
<profiles>
<profile>
<id>test-wip</id>
<properties>
<maven-surefire-plugin.groups>pl.edc.dc.WipTest</maven-surefire-plugin.groups>
</properties>
</profile>
<profile>
<id>verify-wip</id>
<properties>
<maven-failsafe-plugin.groups>pl.edc.dc.WipTest</maven-failsafe-plugin.groups>
</properties>
</profile>
</profiles>
- test-wip — integration tests
- verify-wip — unit tests
Narrows unit tests
$ mvn -P test-wip ..
Narrows integration tests
$ mvn -P verify-wip ..
Narrows unit and integration tests
$ mvn -P test-wip,verify-wip ..
Running tests implicitly inside Maven lifecycle phases
Unit tests
$ mvn test|package
Unit and integration tests
$ mvn verify|install|deploy
Running tests explicitly through Maven plugin goals
Only unit tests
$ mvn surefire:test
Only integration tests
$ mvn failsafe:integration-test
GitHub Actions workflow example
An example application in the GitHub Actions workflow as CI (continuous integration) pipeline. The workflow runs page and pipeline.yml.
All the best, stay healthy and Godspeed!
All trademarks and registered trademarks used or referenced in this article are trademarks of their respective owners.
References
- https://semaphoreci.com/community/tutorials/how-to-split-junit-tests-in-a-continuous-integration-environment
- https://www.mkyong.com/unittest/junit-categories-test
- https://www.javaworld.com/article/2074569/core-java/unit-and-integration-tests-with-maven-and-junit-categories.html
- https://paucls.wordpress.com/2014/02/19/using-junit-categories-to-group-tests
- https://www.testwithspring.com/lesson/running-integration-tests-with-maven
- https://confluence.atlassian.com/clover/using-with-surefire-and-failsafe-plugins-294489218.html
- https://stackoverflow.com/questions/6612344/prevent-unit-tests-but-allow-integration-tests-in-maven
- https://books.sonatype.com/mcookbook/reference/ch07s04.html
- http://tomaszdziurko.com/2013/01/running-unit-tests-integration-tests-separately-maven-testng