Understanding Dependency Management in Maven - Part 1

Introduction

There exists a remarkable file when it comes to Maven — the pom.xml. Developers tend to hate Maven, and in specific this file — if they have no clue about how it works. They do manage, of course. But being aware of every detail of this one file and how it carries out its wonders behind the scenes makes working with Maven a breeze.

pom does a million things — some of which include dependency management, build lifecycle customization and plug-in configuration. This is just the beginning of the list. Yes, this one file has everything!

In today’s blog, I’ll be focusing on every possible detail with respect to one of these aspects — dependency management. This will enable developers to deal with seemingly complicated issues arising from Maven’s dependency management, while also giving a fair idea of how it all works.

The <dependency> tag — a piece of cake

Let’s begin with the easiest part — the <dependency> tag. This is where one would mostly start his/her journey with dependencies in Maven; and the one thing that every beginner who has seen a POM would be knowing (hopefully!). Any dependency (library) that we wish to use in our project, we do so, by first including it under the <dependencies> tag of POM, like this:

<dependencies>
  <dependency>
    <groupId><TheLibrary’sGroupId></groupId>
    <artifactId><TheLibrary’sArtifactId></artifactId>
    <version><TheLibrary’sVersion></version>
  </dependency>
</dependencies>

Or in some cases, just:

<dependencies> 
     <dependency> 
        <groupId><TheLibrary’sGroupId></groupId> 
        <artifactId><TheLibrary’sArtifactId></artifactId> 
     </dependency> 
</dependencies>

It is easy to understand what the <dependencies> tag does — Maven checks if the artifacts (jars mostly) of the libraries specified under this tag are available in our local repository; and if not, it takes the onus of downloading these artifacts from the remote repository into our local one. But wait. What is a local and remote repository?

Secondly, as mentioned in the Maven documentation here, the <version> of each of the included dependencies needs to be specified in the POM. But in the second example shown above, how does it work without doing so?

We’ll be touching upon each of these points in this blog.

Remote Repositories

It is obvious that for a project to be compiled locally, all its dependencies, that is libraries it is dependent on, need to be present locally. Most of the dependencies we generally use are external dependencies, which means that these libraries have been developed by someone else. Where are these external dependencies present? In the remote repository. Thus, remote repository is just a place where the artifacts or jars of these dependencies are hosted, in order to make it accessible for the public to download and use. An analogy being — the PlayStore, from where anyone can download apps built and published by others.

And yes, whenever anyone specifies in the POM, under the <dependencies> tag, that they need certain dependencies, Maven downloads the artifacts of those dependencies from this remote repository.

Where exactly is the remote repository, where the artifacts have been hosted, located? To answer this, we introduce another tag used in the pom.xml — <repositories>, the tag under which the URLs of the remote repositories are configured.

Now, you are scrolling through your POM searching for the <repositories> tag. But no matter how much you try, you are unable to find such a tag in your POM. Yet, the dependencies were downloaded successfully when you built your project. How’s that? Did those libraries get downloaded from nowhere?

Maven, by default, uses the well-known remote repository — the Maven Central repository — (https://repo.maven.apache.org/maven2/) — to locate the artifacts of our dependencies. If all the dependencies we require are present in this Maven Central repository, we don’t need to use the <repositories> tag. However, if the dependencies you wish to use aren’t present in this central repository, say they are libraries internal to your organization and which are thus hosted only on your company’s internal repository manager, you will need to add the address of that repository manager under the <repositories> tag.

Example:

<repositories>
  <repository>
    <id>company’s-internal-repo-manager</id>
    <url>https://company-server/repo</url>
  </repository>
</repositories>

The purpose of repository managers is to cache artifacts from the Central Maven, reducing load on it and at the same time bringing down the time required to download these artifacts. It also serves the objective of hosting company-specific libraries that can’t be hosted in public repositories. A few famous repository managers are JFrog Artifactory and Sonatype Nexus. These platforms host not only Maven JARs, but a plethora of totally unrelated artifacts and binaries, such as container (e.g., docker) images, configuration files and much more.

Bonus point — Since you have made it this far, one interesting fact for you! We mentioned that Maven uses the Maven Central repository as the remote repository by default. But where is this default configured? How does our Maven project understand these default configurations when that’s not been specified anywhere inside our POM?

The Super POM.

All POMs inherit the Super POM implicitly. Find the Super POM here. Notice how the Central Maven repository URL has been specified under the <repositories> tag of the Super POM. Ta-da!

The Super POM has not only the default repositories configured, but several other configurations too, which you can explore on your own by taking a look at the file in the link above.

Local Repositories

Let’s continue.

It’s obvious that to download dependencies from the remote repository every time it is added as a dependency to a project is not feasible. Several libraries are used commonly in multiple projects, and thus instead of downloading them into each individual project (as is done in the case of frameworks such as Node.js where npm downloads libraries into a node_modules directory of every application, though other applications might already be using the same library, unless the library is installed globally), Maven manages this efficiently!

And this is done through the local repository — a name for that central folder in your local system, where the downloaded artifacts of Maven libraries (both ‘external’ dependencies and those created by us) reside. This is the central place in your system where you’ll find all the Maven libraries' artifacts that one of your projects used at a point. Thus, it acts like a cache (similar to a ‘repository manager’). When we add a dependency and build the project with Maven (I won’t be covering the Maven lifecycle commands in this blog; this blog exclusively focusses on dependency management), Maven first checks if the specified version of the library is present in our ‘local repository’. If yes, we just saved a lot of time and network in not having to download the same dependency again from the remote repository! And if not, it downloads that dependency into the local repository, thus removing the need to download it the next time.

So, the idea is simple. ‘Local repository’ is just a repository of artifacts (you can say a ‘local copy’) downloaded from the ‘remote repository’.

And do note — this being the central folder, the library artifacts present here, can be used by not just the one project we built, but every project we built or will be building locally in the future and which uses the same dependency.

Where do I find this local repository?

In ~/.m2 (in Linux & Mac). Thus, the local repository is popularly known as the .m2 directory.

And the directory structure of this repository is again interesting. If you consider the following dependency:

<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-circuitbreaker</artifactId>
<version>1.7.1</versionId>

The jar for this library would be present at the following path — ~/.m2/repository/io/github/resilience4j/resilience4j-circuitbreaker/1.7.1/

To summarize whatever we just read — whenever you need to add a new dependency/library to your project, add it to <dependencies> in the POM -> run mvn compile in the directory containing the POM -> maven checks if the specific version of the artifact is already present in the .m2 directory a.k.a. the local repository -> If yes, great! Nothing more is to be done. If not, it checks the <repositories> tag in order to connect to and download the library artifact from the remote repository into the local one -> Done! Go ahead and import the library in your application and begin using it.

Phew, that was not actually tough.

But that’s just the prelude! There’s a lot more (fortunately).

Wrapping up Part 1

Originally I had planned to include both Part 1 and Part 2 as part of the same blog. However, due to the size of the entire content, I decided to split it into two.

So, do proceed to Part 2 of this blog — it has much more interesting content on dependency management, the BOM, parent POM and more.

See you there!