Migration from C++ native-maven-plugin to an Artifactory.cmake module

For many years we have used an Apache maven native-maven-plugin as part of our internal workflow when building our native (C/C++) modules/libraries. This plugin is out-dated and is unsuitable for more modern forms of build creation. In order to maintain our workflow and continue to use the existing a JFrog Artifactory we sought an alternative plugin to the Apache maven native-maven-plugin. We found, tested and deployed CMake. CMake is a modern build tool which provides all of the functionality we require, it also fits well into our existing workflow and supports the existing Artifactory (a precondition for our original selection process). The use of CMake exceeded our expectations as it — not only — performed as expected but even improved deployment speed by a factor of four! This massive improvement in build speed is due to the newly available parallelization built into CMake. Parallelization of build is not supported by native-maven-plugin.

Motivation

Our existing workflow for build creation was well established and stable. However, it relied on outdated plugins and was slow and cumbersome. We looked for a new modern IDE for native modules. As we looked for alternatives to the existing plugin we were constrained by certain requirements, such as the need to support the existing Artifactory and provide support for C++ modules. These requirements for the new plugin limited the number of potential options available.

Our deployment

Our company generates our product distribution as an ISO file created from a collection of RPM artifacts that are stored in a JFrog Artifactory. This ISO contains C++ libraries as they are required by other modules. In the past we used an Apache maven native-maven-plugin in order to build our C++ based modules. This Apache maven plugin provided full support for all of the tasks required to work within a Java environment. This setup worked well as it allowed the company to generate a product distribution as an ISO file created from a collection of RPM artifacts stored in a JFrog Artifactory. The process was slow and cumbersome and we looked for an alternative which could perform the following tasks:

• utilize an identical build process for all modules (mvn package)
• support Artifactory deployment and dependencies resolution

We decided to implement and trial the use of CMake. We decided to migrate to CMake as it is a more modern and all encompassing solution. The implementation process was not, however, as straightforward as we had hoped. We had to find a solution which allowed for the continued use of the existing Artifactory. This is due to the fact that the whole building process depends on this Artifactory.

After researching several options we decided that the CMake module (available at https://github.com/theatricshrink/Artifactory.cmake) looked like an ideal candidate for implementation.

During the adoption of the module we encountered several issues (or perhaps features) that blocked our current method of usage. In order to overcome these minor challenges we forked the repository and made several changes. These changes are available at https://github.com/backglite/Artifactory.cmake.

Solution

Here we present the usage of the module according to our specific needs. This example is based on one of our library modules that depends on other artifacts and itself is also providing an artifact to be used later on by other modules. In the example provided below we call it our_lib.

For those interested in trying out a similar process we have documented the entire process below.

The dependencies looks like this:

• 3-rd party libraries (other_lib1, other_lib2) ← our_lib ← additional module

In the maven-native-plugin definition of the library build the dependencies were as follows:

Maven Deployment

When we call the mvn deploy, Maven performs the following actions:

1. Downloads four artifacts: other_lib1.a, other_lib1.inczip, other_lib2.a, other_lib2.inczip
2. Compiles our_lib using the included .inczip artifacts and links it with the provided .a artifacts.
3. Produces two artifacts: our_lib.so, out_lib.inczip
4. Deploys the target artifacts to the Artifactory.

CMake Deployment

When migrating to the CMake module, we deployed the library binary to the Artifactory and included one tgz file with the following structure ( lib_name.tgz):

lib_name
 - include
   - foo.h
   - bar.h
 - lib
   - lib_name.a

When using the Artifactory.cmake module, we prepared several helper CMake functions:

Additional Libraries

We then proceeded to download several libraries and added them as dependencies to be used. This was easily performed using our CMakeList.txt:

As a result of the additional libraries some modification was necessary. Whenever we run the cmake build it performs the following additional tasks:

1. forces the download of the .tgz file for both other libraries
2. loads the CMake dependency library based on their binary
3. includes and compiles our own library while using the additional libraries

Final Step

The final step was to deploy our build result into the Artifactory. We prepared our tgz package and deployed it to using the Artifactory.cmake module.

Then we run the following command:

cmake --build ./build --target artifactory-submit

As a result the final artifact our_lib.tgz was uploaded to the Artifactory.

Results- Build speed improvements as a side effect of process improvement.

The end result of the process described above is that we are now able to migrate all our C++ modules to the CMake build while still using Artifactory for artifact storage and dependency management.

As a side effect of this process we have seen a significant improvement in the build speed. It is roughly three to four times faster than previously. We have found that the originally used native-maven-plugin was not able to process C++ builds in parallel. Through the implementation of CMake we are able to pass parallel build parameters to the compiler using the command:

cmake3 --build ./build --target all -- -j `nproc`

A relatively simple change has had a significant impact on our build speed.