Maven Phase Testing COMBO — Cheat Sheet

Tomasz Krzysztof Zieleniewski
EDC Dev Chapter
Published in
4 min readJan 28, 2020

The full cheat sheet source code is available on GitHub.

Small, seemingly insignificant configurations and settings of basic tools of our daily work. Usually, we don’t introduce them too often, so some people, including me, may easily forget them. With the help come cheat sheets, thanks to which we immediately find what we need.
This article contains the cheat sheet for optimal tests configuration and execution in Apache Maven.

Featuring with

  • Enables multilevel testing (“TestPyramid”), where tests are split into two groups, low-level unit tests and high-level integration tests
  • Applies and simultaneously handles JUnit 4, JUnit 5 and TestNG frameworks
  • There is a possibility to execute tests, from one or both groups, in the failsafe mode
  • There is a possibility to exclude tests, from one or both groups
  • There is a possibility to narrow tests, from one or both groups, to WIP tests

Preparing Maven

Tests are split into two groups which are executed in separate Maven phases. An execution in separate plugins enables independent management of each test group.

<build>
<plugins>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
</plugin>
</plugins>
</build>

Multiple tests frameworks support

In order to detect and process distinct tests frameworks, dedicated frameworks providers need to be specified explicitly as plugins dependencies (the same dependencies in both plugins).

Additionally there is a need to exclude JUnit tests handling from TestNG provider, which by default supports JUnit as well.

<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit-platform</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-testng</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
</dependencies>
<configuration>
<properties>
<!-- disables JUnit tests for TestNG provider -->
<property>
<name>junit</name>
<value>false</value>
</property>
</properties>
</plugin>

Marking tests

Unit and integration tests are marked and separated on the basis of specific naming patterns. There is no additional configuration needed.

Unit tests names need to be compliant with the default Maven Surefire Plugin includes pattern.

Integration tests names need to be compliant with the default Maven Failesafe Plugin includes pattern.

Exemplary unit test

class ExampleTest {
..
}

Exemplary integration test

class ExampleIT {
..
}

Failsafe mode

Enabled through common plugins configuration property <testFailureIgnore>.

<properties>
<testFailureIgnore>false</testFailureIgnore>
<testFailureIgnoreITs>${testFailureIgnore}</testFailureIgnoreITs>
<testFailureIgnoreUTs>${testFailureIgnore}</testFailureIgnoreUTs>
</properties>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<testFailureIgnore>${testFailureIgnoreITs}</testFailureIgnore>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<testFailureIgnore>${testFailureIgnoreUTs}</testFailureIgnore>
</configuration>
</plugin>

Set through custom Java system properties. By default executions are not failsafe.

  • -DtestFailureIgnoreITs — ignores failures for integration tests
  • -DtestFailureIgnoreUTs — ignores failures for unit tests
  • -DtestFailureIgnore — ignores failures for unit and integration tests

For unit tests

$ mvn -DtestFailureIgnoreUTs ..

For integration tests

$ mvn -DtestFailureIgnoreITs ..

For unit and integration tests

$ mvn -DtestFailureIgnore ..

Exclusion

Enabled through plugins configuration properties.

<properties>
<skipITs>${skipTests}</skipITs>
<skipUTs>${skipTests}</skipUTs>
</properties>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<skipTests>${skipUTs}</skipTests>
</testFailureIgnore>
</configuration>
</plugin>

Set through Java system properties. By default there are no executions.

  • -DskipITs — skips integration tests (Maven Failsafe Plugin property)
  • -DskipUTs — skips unit tests (custom property)
  • -DskipTests — skips unit and integration tests (Maven Surefire Plugin property)

Skips unit tests

$ mvn -DskipUTs ..

Skips integration tests

$ mvn -DskipITs ..

WIP narrowing

There is a need to configure plugins groups property and test needs to be marked by an annotation.

<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<configuration>
<groups>${maven-failsafe-plugin.groups}</groups>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<groups>${maven-surefire-plugin.groups}</groups>
</configuration>
</plugin>
  • JUnit 4 tests marked by the class and method level @Category annotation.
import org.junit.experimental.categories.Category;
import pl.edc.dc.UnitTest;
@Category(WipTest.class)
public class JUnit4Test {
..
}
  • JUnit 5 tests marked by the class and method level @Tag annotation.
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
@Tag("pl.edc.dc.WipTest")
class JUnit5Test {
..
}
  • TestNG test marked by the groups attribute of the class and method level @Test annotation.
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Test(groups = {"pl.edc.dc.WipTest"})
public class TngTest {
..
}

WIP narrowing is activated through a dedicated Maven build profile for the given testing phase.

<profiles>
<profile>
<id>test-wip</id>
<properties>
<maven-surefire-plugin.groups>pl.edc.dc.WipTest</maven-surefire-plugin.groups>
</properties>
</profile>
<profile>
<id>verify-wip</id>
<properties>
<maven-failsafe-plugin.groups>pl.edc.dc.WipTest</maven-failsafe-plugin.groups>
</properties>
</profile>
</profiles>
  • test-wip — integration tests
  • verify-wip — unit tests

Narrows unit tests

$ mvn -P test-wip ..

Narrows integration tests

$ mvn -P verify-wip ..

Narrows unit and integration tests

$ mvn -P test-wip,verify-wip ..

Running tests implicitly inside Maven lifecycle phases

Unit tests

$ mvn test|package

Unit and integration tests

$ mvn verify|install|deploy

Running tests explicitly through Maven plugin goals

Only unit tests

$ mvn surefire:test

Only integration tests

$ mvn failsafe:integration-test

GitHub Actions workflow example

An example application in the GitHub Actions workflow as CI (continuous integration) pipeline. The workflow runs page and pipeline.yml.

All the best, stay healthy and Godspeed!

--

--

Tomasz Krzysztof Zieleniewski
EDC Dev Chapter

Software Engineer 👊 Team Lead @work> quality never goes out style