Keeping Dependencies Up To Date with Maven-Centralized Dependency Management

Credit to Олександр Edelvejs

Normally, when you start a project, you set your required dependencies up with the latest stable versions of all libraries and plugins.

Then time goes by, the project grows, and new features and libraries are added. But the versions of the 3rd-party dependencies and the plugins remains the same; the team never updates them.

Now, that’s all fine and dandy until… there׳s a conflict. If each team has its own repository, how can dependencies be managed and conflict frequency reduced?

What is dependency conflict

The dependency issue arises when multiple libraries have same dependencies (internal / 3rd-party) on with different versions, the shared library can only be installed with a single version, thus causing version conflict.
Conflicting versions of 3rd-party libraries known as well as our own- appearing in production as ‘NoSuchMethodError’ or ‘ClassNotFoundError’.

Repository structure

Some time back at Outbrain, we had made a transition from one big mono-repo to a multi-repo approach that required all teams to manage their dependencies by themselves, reducing visibility and control of the dependencies among multi-repos. Unfortunately, this gave rise to dependency conflicts caused by conflicting versions of 3rd-party libraries. (To read more about Outbrain’s approach to repository structures and version conflicts, check out Mono-repo vs Multi-repo vs Hybrid: What’s the Right Approach?’’

After some deliberation on how to proceed, we decided on a hybrid approach: We have one mono-repo that is responsible for keeping internal shared libraries, versions of 3rd-party dependencies, and APIs among teams and that can be released and managed by a specific version. The teams, however, each use their own multi-repos for a service code that uses library API’s, which are managed by a property version that was part of the mono-repo release process.

So, to use the latest internal API, we just need to upgrade specific property versions in the multi-repo’s.

Can 3rd-party dependencies be upgraded just as easily? Yes!

Centralized dependency management

The mono-repos should reduce the frequency of dependency conflicts, since we have one place with a repo that manages all dependencies.

How is conflict frequency reduced?
All the internal shared libraries are compiled and released with the same dependencies because they are all defined under the same pom.xml (the parent pom.xml).

The mono-repo has a module called “service-pom” with only one pom.xml file that contains the common plugin definitions and all the 3rd-party versions for all the deployable services. The service repos (multi-repos) must be inherited from the service-pom. In other words, the entire service-repo gets all the relevant properties from one place- the centralized dependency management. This service-pom is released with a specific version as part of the mono-repo release process.

Pom hierarchy in the mono-repo
This repo contains all our shared libraries. As part of this repository we have modules with a structured pom hierarchy for pom’s that are part of the mono-repo release process.

A drill down into the pom hierarchy
The service-pom
This service-pom module contains only one pom.xml and is meant to be used as a parent pom for all service repos. In the image below, we can see how a service pom looks.

<project>
<parent>
<groupId>com.outbrain</groupId>
<artifactId>version-pom</artifactId>
<relativePath>../version-pom</relativePath>
<version>${revision}</version>
</parent>
<groupId>com.outbrain</groupId>
<artifactId>service-pom</artifactId>
<name>Common to all services</name>
<version>${revision}</version>

<build>
<pluginManagement>
<plugins>
...more...
</plugins>
</pluginManagement>
</build>
</project>

The version-pom
This pom defines all the 3rd-party versions that we use. Below, we can see how the 3rd-party definitions are organized.

<project>
<groupId>com.outbrain</groupId>
<artifactId>version-pom</artifactId>
<version>${revision}</version>

<properties>
<revision>5.0.0</revision>
<libs.version>${revision}</libs.version>
<log4j2.version>2.17.0</log4j2.version>
...more...
...more...
</properties>

<dependencyManagement>
<dependencies>
<!-- 3rd party artifacts definitions -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j2.version}</version>
</dependency>
...more...
...more...
</dependencies>
</dependencyManagement>
</project>

The module-pom
This pom aggregates all the needs of swinfra-libs modules and is used as a bom pom. ּּּּּּּBelow is a layout of the internal-library definitions.

<project>
<parent>
<groupId>com.outbrain</groupId>
<artifactId>version-pom</artifactId>
<relativePath>../version-pom</relativePath>
<version>${revision}</version>
</parent>
<groupId>com.outbrain</groupId>
<artifactId>modules-pom</artifactId>
<version>${revision}</version>

<dependencyManagement>
<dependencies>
<!-- Internal shared libs definitions -->
<dependency>
<groupId>com.outbrain</groupId>
<artifactId>mono-repo-lib1</artifactId>
<version>${libs.version}</version>
</dependency>
<dependency>
<groupId>com.outbrain</groupId>
<artifactId>mono-repo-lib2</artifactId>
<version>${libs.version}</version>
</dependency>
...more...
...more...
</dependencyManagement>
</dependencies>
</project>

Service-repos
All service repositories’ repo projects must:

1. inherit from the service-pom, and
2. add modules-pom as a bom dependency

In the image below, we have service-repo pom.xml definitions.

<project>
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.outbrain</groupId>
<artifactId>service-pom</artifactId>
<version>2.0.0</version>
</parent>
<artifactId>service-repo-one</artifactId>
...
<dependencies>
<dependency>
<groupId>com.outbrain</groupId>
<artifactId>modules-pom</artifactId>
<version>${bom.revision}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
.....
</dependencies>
</project>

As mentioned earlier, the service-pom that was released and managed by the version contains all the 3rd-party dependencies with the specific versions, since all our service-repos inherit from the service-pom (as is shown in the image above). Now, to upgrade 3rd-party dependencies, we can work in the mono-repo (simultaneously upgrading multiple dependencies, which will be released and managed by only one version as part of the mono-repo release process). Consequently, we can upgrade the service-pom across all the multi repos at once.

Upgrade multiple dependencies, by managing one version
As mentioned earlier, the mono-repo can be released multiple times a day and contain the code changes of shared libraries and/or multiple 3rd-party upgrades, with all these changes being released into one version.
So, all we need to update is the service-pom version across all our multi-repos.

Manual upgrades? Who needs them?
We have an automatic tool called “Bumper” that, at a fixed frequency, upgrades and creates pull requests across all service repos with the latest version of the released version, and then merges them. Afterwards, the services are automatically deployed with CI/CD pipelines . When we needed to tackle the Log4j Vulnerabilities, for example, the centralized dependency management and the bumper tool took care of everything.

Got 200 repositories to update? Just head to the break room while the maven-centralized dependency management and bumper tool to do it all for you.

--

--

--

Outbrain is the world’s leading native advertising platform, guiding the digital discoveries of consumers around the globe. Genuinely connecting marketers, publishers, and the consumers in-between, Outbrain serves more than 308 billion recommendations, organically personalizing,

Recommended from Medium

Flutter Secure your sensitive data

Containers and microservices

Validating critical email address in Laravel

M1 Mac: Combating “x86_64 architecture is required for this software.”(A Python 3.7 Example)

Industry Use cases of Open shift

Get The Most Out Of A Tiny Bug

How Dropbox used AWS when it was started as a start-up?

Implementing GDPR — a Product Manager’s Perspective

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Avi Youkhananov

Avi Youkhananov

I’m an application software engineer at Outbrain, with a passion for new technology.

More from Medium

Manage libraries, artifacts and deliverables with Nexus Repository Manager OSS

Docker updates terms of service for Docker Desktop!

Dapr and Kafka-easy binding

Kicking it with Kubernetes ☄️