Maven For Pipelining Part 2

This is part two of a three-part series describing how we use Maven as a build tool in CICD pipelines. The first part introduced our approach to using Maven and showcased a parent POM that sets up the Maven Lifecycle. This part focuses on a child project and presents the individual Maven commands used for each pipeline step. The third part concludes the series with a discussion on test design and showcases a fully functional HTTP API.

Now that we have a solid parent POM which provides all of our Maven Lifecycle functionality (see part 1), it is time to look at a bare-bones child JAR project. You can clone the project from GitHub.

$ git clone https://github.com/eonian-technologies/example-child-jar.git

The project’s POM is super simple. It only contains its own direct dependencies. But when running mvn install, it still executes the plugins that are bound to lifecycle phases in the parent POM.

$ mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: JAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean)...
...
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-blacklist)
...
[INFO] --- git-commit-id-plugin:2.2.2:revision (set-git-properties) 
...
[INFO] --- maven-antrun-plugin:1.8:run (generate-build-properties)
...
[INFO] --- maven-resources-plugin:3.0.2:resources...
...
[INFO] --- maven-compiler-plugin:3.6.1:compile...
...
[INFO] --- maven-resources-plugin:3.0.2:testResources...
...
[INFO] --- maven-compiler-plugin:3.6.1:testCompile...
...
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent...
...
[INFO] --- maven-surefire-plugin:2.20.1:test (default-test)
...
[INFO] --- jacoco-maven-plugin:0.7.9:report...
...
[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar)...
...
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration...
...
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test...
...
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration...
...
[INFO] --- maven-failsafe-plugin:2.20.1:verify...
...
[INFO] --- maven-install-plugin:2.4:install...
...
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 2.248 s
[INFO] Finished at: 2017-07-15T17:22:18-04:00
[INFO] Final Memory: 21M/256M
[INFO] -------------------------------------------------------------

As we discussed in part 1, executing a lifecycle phase like mvn install is great for local and IDE development, but when we pipeline, we want to execute discrete pipeline steps. Here are the basic CI steps of a CICD pipeline.

Build
Unit Test
Integration Test
Code Analysis
Publish Artifacts

Build

The first step in our CI pipeline is to build the project. But this is not just compiling the code. We’ll also want to enforce our dependency blacklist, create the build.properties file, and do all the things that the Maven Lifecycle does to process resources, class files, and dependencies (see part 1 for details). The easiest way to do this is to use the verify goal from the Maven Lifecycle and skip all testing.

$ mvn -DskipTests clean verify
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: JAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean)...
...
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-blacklist)..
...
[INFO] --- git-commit-id-plugin:2.2.2:revision (set-git-properties)
...
[INFO] --- maven-antrun-plugin:1.8:run (generate-build-properties)..
...
[INFO] --- maven-resources-plugin:3.0.2:resources
...
[INFO] --- maven-compiler-plugin:3.6.1:compile
...
[INFO] --- maven-resources-plugin:3.0.2:testResources
...
[INFO] --- maven-compiler-plugin:3.6.1:testCompile
...
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent...
[INFO] Skipping JaCoCo execution...
...
[INFO] --- maven-surefire-plugin:2.20.1:test
[INFO] Tests are skipped.
...
[INFO] --- jacoco-maven-plugin:0.7.9:report
[INFO] Skipping JaCoCo execution...
...
[INFO] --- maven-jar-plugin:3.0.2:jar
...
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration
[INFO] Skipping JaCoCo execution...
...
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test...
[INFO] Tests are skipped.
...
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration...
[INFO] Skipping JaCoCo execution...
...
[INFO] --- maven-failsafe-plugin:2.20.1:verify...
[INFO] Tests are skipped.
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 1.765 s
[INFO] Finished at: 2017-07-16T17:14:25-04:00
[INFO] Final Memory: 22M/310M
[INFO] -------------------------------------------------------------

This will run the lifecycle through the verify goal, but will skip the plugins related to testing. The result is that all classes and resources have been processed, the source code has been compiled, and we have executed all plugins that are not test-related. If things went well, we can continue to the next step.

Unit Test

Next we will execute the SureFire plugin to run our unit tests. But we also need to execute the JaCoCo plugin which will determine test coverage and generate the coverage report. When we were executing the Maven Lifecycle directly, the JaCoCo plugin was automatically executed because it was bound to the lifecycle as part of the unit-test profile. But outside the lifecycle, we’ll need to specify each plugin we want to run.

$ mvn jacoco:prepare-agent@preTest surefire:test jacoco:report@postTest
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: JAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (preTest)...
...
[INFO]
[INFO] --- maven-surefire-plugin:2.20.1:test (default-cli)...
...
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:report (postTest)...
...
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 0.788 s
[INFO] Finished at: 2017-07-16T11:53:40-04:00
[INFO] Final Memory: 14M/245M
[INFO] -------------------------------------------------------------

Integration Test

As discussed in part 1, integration tests are super important for WAR projects that are deployed to a server. But our JAR project might also contain code that requires infrastructure. If so, we’ll need to write integration tests and execute them similarly to the way we did the unit tests — calling the plugins directly.

NOTE: In this example, our code consumes a static development environment when we run the tests. For example, if our code needs to connect to a database, it connects to the dev database. Setting up the code to be “environment aware” is an important part of pipelining. We will discuss static environments and portable code in upcoming articles.

$ mvn jacoco:prepare-agent-integration@preIT failsafe:integration-test jacoco:report-integration@postIT failsafe:verify
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: JAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration (preIT)
...
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-cli)
...
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration (postIT)
...
[INFO] --- maven-failsafe-plugin:2.20.1:verify (default-cli)
...
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 0.722 s
[INFO] Finished at: 2017-07-16T17:01:31-04:00
[INFO] Final Memory: 14M/309M
[INFO] -------------------------------------------------------------

Integration Test — WAR Project

The previous output is from our JAR example. Let’s now take a look at how our parent POM handles integration tests for a child WAR project. You can clone the project from GitHub.

$ git clone https://github.com/eonian-technologies/example-child-war.git

As discussed in part 1, we’ll spin up a local server and deploy the project before running integration tests. The server is configured to measure test coverage, and the FailSafe plugin is used to run the tests. Tests are written make calls to the local server, and then validate the response of each call. After the tests have run, coverage information is dumped, the server is stopped, and a coverage report is generated. As with the JAR project, the test run is then validated, breaking the build if there were failed tests.

NOTE: As with our JAR project, when the integration test server starts, our WAR project will connect to resources in a static development environment.

$ mvn jacoco:prepare-agent-integration@preIT cargo:start@preIT failsafe:integration-test jacoco:dump@postIT cargo:stop@postIT jacoco:report-integration@postIT failsafe:verify
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: WAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration (preIT)
...
[INFO] --- cargo-maven2-plugin:1.6.2:start (preIT)
...
[INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-cli)
...
[INFO] --- jacoco-maven-plugin:0.7.9:dump (postIT)
...
[INFO] --- cargo-maven2-plugin:1.6.2:stop (postIT)
...
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration (postIT)
...
[INFO] --- maven-failsafe-plugin:2.20.1:verify (default-cli)
...
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 10.372 s
[INFO] Finished at: 2017-07-17T15:23:30-04:00
[INFO] Final Memory: 17M/309M
[INFO] -------------------------------------------------------------

If you do not want to use the FailSafe plugin to run your integration tests, you can easily bind Newman, JMeter, or Selenium to the integration-test phase in your child project. We will showcase this in later articles.

Let’s return to our JAR project.

Code Analysis

Now that we have run unit tests and integration tests we can determine the overall test coverage and run static analysis. This is an important step. We’ll need to ensure that the build passes the quality gates we’ve set up in our SonarQube server. The gates should include checks on code quality and ensure that the test coverage meets the minimum requirements.

$ mvn sonar:sonar -Dsonar.host.url=URL -Dsonar.login=TOKEN
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: JAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- sonar-maven-plugin:3.2:sonar (default-cli)...
...
[INFO] Load project repositories (done) | time=25ms
[INFO] Load quality profiles
[INFO] Load quality profiles (done) | time=49ms
[INFO] Load active rules
[INFO] Load active rules (done) | time=842ms
...
[INFO] Load server rules
[INFO] Load server rules (done) | time=167ms
...
[INFO] ANALYSIS SUCCESSFUL
[INFO] Task total time: 2.860 s
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 4.633 s
[INFO] Finished at: 2017-07-16T17:39:11-04:00
[INFO] Final Memory: 37M/553M
[INFO] -------------------------------------------------------------

Publish Artifacts

At this point we are ready to publish the build artifact. Like before we’ll explicitly call the deploy plugin. But because we are calling the plugin outside of the Maven Lifecycle, Maven does not know about the artifact that was previously built. So we’ll need to specify the build artifact, the POM file, the URL to our artifact repository, and the ID of the repo in the settings.xml file that contains the repository credentials.

$ mvn deploy:deploy-file -DpomFile=pom.xml -Dfile=target/ARTIFACT -Durl=REPO_URL -DrepositoryId=REPO_ID
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------------------------------------------
[INFO] Building Example: JAR Project 1.0-SNAPSHOT
[INFO] -------------------------------------------------------------
[INFO]
[INFO] --- maven-deploy-plugin:2.8.2:deploy-file (default-cli)...
Downloading:...
Uploading:...
Uploading:...
[INFO] -------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] -------------------------------------------------------------
[INFO] Total time: 2.194 s
[INFO] Finished at: 2017-07-16T18:06:26-04:00
[INFO] Final Memory: 12M/205M
[INFO] -------------------------------------------------------------

That’s it. We’ve successfully broken the Maven Lifecycle into discrete pipeline steps, and we did so with very little effort. As long as we execute these steps in order, and only if the previous step succeeds, our pipeline will work as intended. Later articles will discuss how to build the CI jobs which run these commands, but this effort sets us up nicely for when we get there. Ensuring pipeline steps can be run by our build tool, from the command line, is the very first step.

Next…

Take some time to play with the JAR and WAR projects and be sure to read through all the POM files before continuing to part 3, where I’ll conclude the series with a discussion on test design and showcase a fully functional HTTP API.

ABOUT THE AUTHOR:
Michael Andrews is an experienced platform/cloud engineer with a passion for elegant software design and deployment automation. He is committed to developing light-weight malleable software and decoupled event-driven code using Domain-driven Design principles and hexagonal architecture. He is a specialist in Kubernetes, Java, Spring, DevOps, CICD, and fully tested low risk no downtime automated deployments.