Building dependent Maven projects

8 min readSep 24, 2018

The Kata

Recently, I was solving a little CI kata about building the dependent Maven projects. Typically, if multiple Maven projects are developed simultaneously, it is a good practice to place those under a common parent and commit to the same VCS repository. However, what if those projects reside in different VCS repositories? Also, what if the development is done in branches?

I assume here that branch naming is used to connect the modules located in the different VCS repositories logically. If we are building the master branch for the Application module, we depend on the Component module that should also be built from the master branch. If we are building Application from branch ‘feature-001’, then Component should also be built from ‘feature-001’, etc.

Usually, in a situation like this one, we can build the Component, deploy the binary to a Maven repository, and then build the Application.

However, in the case with branches we can easily create an inconsistent situation: if we build the Application, there is no guarantee that we pull in the Component that is built from the correct branch.

“Source code repositories have branches, binary ones don’t. So any part of your development plan that relies on branching can’t rely on binary repositories for things that are in the scope of branches.” — discussion at StackExchange

The Drama. Looking for a solution

Okay, what if I use the version of the Component to identify the branch? The version “1.0-SNAPSHOT-master” identifies the “1.0-SNAPSHOT” built from the master branch.

Using the Maven classifiers

One thing that comes to my mind when I want to have a prefix in the version name is the Maven classifier. Maybe we can use that for our task?

“The classifier distinguishes artifacts that were built from the same POM but differ in content.” — Maven documentation.

Maybe that’s what we need? Perhaps, I could configure maven-deploy-plugin to add the classifier?

<plugin>
  <version>2.8.2</version>
  <artifactId>maven-deploy-plugin</artifactId>
  <executions>
    <execution>
      <phase>deploy</phase>
      <goals>
        <goal>deploy</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <classifier>${project.vcs.branch}</classifier>
  </configuration>
</plugin>

Run mvn -Dproject.vcs.branch=master deploy, and… nope. That doesn’t work. Even though the schema allows adding the classifier element to the configuration, it doesn’t have any effect.

Okay, what if I add the classifier to maven-jar-plugin and produce the artifact with the desired prefix when packaging the archive.

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <version>3.1.0</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>jar</goal>
            </goals>
            <configuration>
                <classifier>${project.vcs.branch}</classifier>
            </configuration>
        </execution>
    </executions>
</plugin>

Run mvn -Dproject.vcs.branch=master clean package, and… it produces two artifacts: component-1.0-SNAPSHOT.jar and component-1.0-SNAPSHOT-master.jar. So if I run mvn deploy next, it deploys 2 files to the repository and that’s not exactly what we want.

You have to use a classifier to attach supplemental artifacts to the project instead of replacing them. Hence, we have to look for something else.

Maven Git versioning extension

The Maven Git versioning extension looks promising! I’ve configured the extension using the example at GitHub and voila! When I run mvn clean package deploy, it produces and deploys just component-1.0-SNAPSHOT-master.jar — awesome!

The most awesome thing is that pom.xml stays clean — no need to add any properties. It also means that pom.xml stays the same in all the branches (unless you start changing it yourself for some other purpose).

Okay, now I produced the artifact with the branch identifier. The next build should now be able to consume this special version. We could, in theory, hardcode the version of the dependency, as 1.0-SNAPSHOT-master. Or even we could use a classifier for the dependency…

However, the above means we have to pass in the property via -D that would identify the branch for the dependency. It would have been much more elegant if there was an extension where we could configure, for which dependencies the version should be modified with the branch name. The project’s pom.xml would stay clean without the need to pass in an extra property to identify the branch.

So, the maven-git-versioning-extension is awesome but almost useless for the dependent builds.

Using a property

Okay. I’m thinking I should just make it work. Using properties seems like the one thing that works for sure.

In the Component’s pom.xml I specify the property placeholder in the version name:

<version>1.0${project.vcs.branch}-SNAPSHOT</version>

And declared the property itself:

<properties>
  <project.vcs.branch/>
</properties>

Now I have to always provide the parameter: mvn -Dproject.vcs.branch=-master clean package. The solution is not too elegant as I always have to provide the parameter, but it works.

For the Application module, I can then specify the dependency as follows:

<properties>
  <component.branch/>
</properties>...<dependency>
  <groupId>org.arhan</groupId>
  <artifactId>component</artifactId>
  <version>1.0${component.branch}-SNAPSHOT</version>
</dependency>

Run mvn -Dcomponent.branch=-master clean package for the Application and it pulls in the desired version of the Component for this build. Ugly, but it works.

Now it is easy to create a build chain in TeamCity, configure the feature branches, and add the branch name as an additional argument via %teamcity.build.branch% parameter.

It all does not work!

In fact, it is all very naive. You can’t make it work consistently as long as the external Maven repository is involved.

For instance, we build the Component and deploy an artifact to the external Maven repository. The Application builds should download that same binary. However, there’s an undefined period when the Application build starts.

While we are waiting for the build, some other build pipeline can update the SNAPSHOT. Then the Application module builds against the latest version of the dependency. This may result in build “flickering”, i.e. the dependent build may fail, but it can also succeed.

In most cases, we get away with this issue because there’s probably not so many builds are updating the same artifact. However, at scale, it might quickly become a PITA.

Chasing the consistency

Perhaps, we could build everything in one run: build the first artifact (Component), install it into the local Maven repository, and run the second build (Application) right after.

However, in this case, we would lose all the benefits of re-using the build results. For instance, if there a more than one dependent project that requires the same dependency. So I still want to find a solution when the builds are separate and could even run on different hosts.

Right before building the Application, the process could download the artifacts from the related Component build, install these binaries into the local Maven repository and run the Application build.

It would be possible to do the same with the external repository if we only could identify the correct binary using metadata. For instance, in JFrog’s Artifactory properties could be used to assign an identifier to the deployed artifact and then use the same identifier in downstream builds to download the correct dependency.

Configuring build dependencies in TeamCity

In TeamCity, we shall have two build configurations — one to build the Component, and the other to build the Application. The Application depends on Component via a snapshot dependency and also an artifact dependency:

The snapshot dependency declares that the two build configurations (one that builds the Component, and the other that builds the Application) are related and form the build chain.

The artifact dependency declares which artifacts (read: files) we need for the given build configuration to work. Before the Application build starts, TeamCity makes sure that the required artifacts are downloaded from the related Component build — this ensures we have the correct version of the dependency.

Installing the dependency to the local Maven repository

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.arhan</groupId>
  <artifactId>install-component</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-install-plugin</artifactId>
        <version>2.5.2</version>
        <executions>
          <execution>
            <id>install-component</id>
            <phase>package</phase>
            <goals>
              <goal>install-file</goal>
            </goals>
            <configuration>
              <groupId>org.arhan</groupId>
              <artifactId>component</artifactId>
              <version>1.0-SNAPSHOT</version>
              <packaging>jar</packaging>
              <createChecksum>true</createChecksum>
              <file>../component-1.0-SNAPSHOT.jar</file>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Running mvn package -f install.xml would install the file into the local Maven repository. I have placed the install.xml into a subdirectory of the project and configured the relative path to the artifact that I want to install (<file>../component-1.0-SNAPSHOT.jar</file>). The working directory has to be set accordingly.

After that, we can start the normal Maven build that consumes artifacts from the local repository.

An important note: for this to work, the setting to use a local artifact repository should be enabled in both the Maven build steps:

Now the build pipeline would build the Component and use the produced binary (component-1.0-SNAPSHOT.jar) to build the Application. The solution will also work correctly in case if we work with branches — TeamCity feature branches feature will help with that.

The only way the dependent build could fail now is when we submit a change to the Component but forget to also push the corresponding change to the Application.

Conclusion

Consistently building the dependent Maven projects isn’t a trivial task. It would have been much easier if Maven supported not just the snapshot versions, but the possibility to use metadata for deploying and the artifacts. Perhaps, in Maven 4? ;)

P.S. TeamCity Kotlin DSL configuration

If you are interested in trying out the sample project, I have uploaded the configuration to a GitHub repository. The .teamcity directory contains Kotlin DSL settings for the pipeline that will build an application and link it with a correct version of the component.

To configure the pipeline in TeamCity, create a project from URL. TeamCity will detect .teamcity folder in the repository and will suggest creating the project using the settings: