My journey to a clear test output in Maven
How I achieved a clear test output with Maven + JUnit5 and how it took me into Open Source
After years of working with this title's stack, I ran my application tests one day, and I finally got pissed. Why are the tests so clean on my IDE, but it looks so much like crap when I run it on the console, thus in any pipeline that gets to build my projects?
How cool would it be if we could run tests and get an output that focuses on the test, making it nice and clear, just like these test results are shown in the IDE? That's what we'll be discussing now.
The default output
The default Maven Surefire output is not focused on the tests. Have you ever tried to run tests on a Java application that runs SpringBoot?
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test)
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------//TONS OF BOOTSTRAPPING LINES . ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.7.1)//TONS LINES ABOUT YOUR APPLICATION BEHAVIOUR, THIRD-PARTY LOGGING, AND STUFF. (FOR EACH ONE OF YOUR TESTS AND/OR CLASSES)
Suppose we were trying to run the standalone application. That would be fine. But we are not. We are running tests. These logs, beyond not contributing to the tests, are making the tests confusing simply because there is too much garbage and boilerplate being printed on the console. If the output makes you wonder whether a test is starting or finishing, then it's not a good output.
Improving the output
Once I worked with unit tests for Node.js, and there I saw one of the clearest test outputs ever, just as if I was running the test inside the IDE.
Mocha output is simple and clean. It has all the details needed and prints the stack trace when a test fails, so it's easy to find out why. You don't get lost inside a bunch of logs. It has descriptive names so everybody can understand what's happening.
That's what I wanted for the Maven Surefire + JUnit5 project I was developing, so I started trying to polish my outputs.
Getting rid of all non-testing-related logs
The most annoying thing in the default output is the bunch of unnecessary logs it prints. The default version is a little difficult to disable. So, it's easier if we override the maven-surefire-plugin version that spring-boot-maven-plugin brings within itself to the version 3.0.0-M4. As soon as we do that, we can finally disable the consoleOutputReporter.
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
<configuration>
<consoleOutputReporter>
<disable>true</disable>
</consoleOutputReporter>
</configuration>
</plugin>
</plugins>
</build>
It is cleaner for sure but still doesn't look like IntelliJ IDEA or Mocha output. I can see the number of tests running, but it's still not possible to see which tests are running, nor more descriptive information about them.
Display more information about the tests that are running
To display more information about the tests, a configuration parameter, disabled by default for some reason, enables all the tests to be logged. This configuration is the reportFormat, which needs to be set to plain.
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
<configuration>
<reportFormat>plain</reportFormat>
<consoleOutputReporter>
<disable>true</disable>
</consoleOutputReporter>
</configuration>
</plugin>
</plugins>
</build>
It's still ugly, but hey! If we compare it with what we had before, it's beautiful, right? No, it's not. It doesn't show nested tests correctly nor the descriptive naming that JUnit5 provides with the DisplayName annotation.
And that is the peak of clearance that Surefire can give us only by configuring its parameters. But, realizing that it could be an issue, the Maven Surefire team created a configuration that enables new extensions for printing tests, be it on console or reports. And so did I.
Use a TreeView Extension
After trying everything possible to make the output clean, I could not go back anymore and had to put more effort into developing this extension. Since nobody had developed any extension before, I didn't have any references, so I needed to get in touch with the Maven Surefire team by e-mail for some guidance, but it worked. I did this, so you just have to add one more configuration to add to your application's pom.xml.
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M7</version>
<dependencies>
<dependency>
<groupId>me.fabriciorby</groupId>
<artifactId>maven-surefire-junit5-tree-reporter</artifactId>
<version>1.1.0</version>
</dependency>
</dependencies>
<configuration>
<reportFormat>plain</reportFormat>
<consoleOutputReporter>
<disable>true</disable>
</consoleOutputReporter>
<statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode"/>
</configuration>
</plugin>
</plugins>
</build>
That's all we ever wanted. DisplayNames, nested tests being printed correctly, some coloring, simple and straightforward. No more getting lost in the pipeline with a bunch of unnecessary testing logs.
Contributing to the project
This extension is my first contribution to the open source community and can be found here. Nowadays, there are 24 projects on GitHub using this dependency. Feel free to check it out, share, find bugs and open an issue for improvements.
Besides, the Maven Surefire team encourages you to create your extension following this page.
Bonus
There’s a standalone application called JUnit5 Console Launcher that was built with the only purpose of running and visualizing JUnit tests on the console. The output is very clean. It will show you the class and test names whenever you make parametrized or nested tests with all the possible details.
The only problem is that the integration with a simple Maven Build is not so easy.
Do you think you have what it takes to be one of us?
At WAES, we are always looking for the best developers and data engineers to help Dutch companies succeed. If you are interested in becoming a part of our team and moving to The Netherlands, look at our open positions here.
WAES publication
Our content creators constantly create new articles about software development, lifestyle, and WAES. So make sure to follow us on Medium to learn more.
Also, make sure to follow us on our social media:
LinkedIn — Instagram — Twitter — YouTube
