Don’t build fat jars for Docker applications

When building Java applications with Maven, like Spring Boot or Vert.x apps, a popular way to go is to bundle the application code and all of its dependency jars into a single fat jar. Typically, the Maven Shade plugin is used for this purpose.

But is this really a good idea? When building a micro service, your application code will be compiled into a few hundred kilobytes or a couple of megabytes of class files. This alone will result into a rather small jar file.

But wait until the shade plugin has finished. It will add the class files of all your dependencies to this jar file. Suddenly, your application file may easily grow to a size of several hundred megabytes.

Inefficiency and heavy load

As Jonathan Haber mentioned in his great article several years ago, one might argue that jars where never meant to be used like this. Furthermore, it is error-prone to aggregate lots of jars into one single file.

Classes with the same name and package might exist among your dependencies. Which one will end up in your fat jar?

The bigger issue in my opinion is inefficiency. If you publish your fat jar into an artifact repository, your bundled dependencies will occupy storage space as individual artifacts while being stored as part of the fat jar as well. That doesn’t make much sense if you ask me.

Building a fat jar is time consuming. Combining the contents of all dependency jars all the while dealing with duplicate class files makes your build take longer than necessary. This is a considerable amount of file I/O your CI infrastructure has to handle on a regular basis, i.e. on each build.

A clever classpath to the rescue

Running a fat jar as a java application is fairly simple:

$ java -jar application.jar

As the fat jar is self contained, this is all you need.

Running a jar file that requires additional jars is more complicated. You cannot specify -jar together with -cp. This will not work:

$ java -jar application.jar -cp \
            lib/dependency1.jar;lib/dependency2.jar

However, a jar file can contain a manifest file that in turn can define a dedicated classpath.

So the solution for us is to build a slim jar file with a proper manifest. The latter having a classpath that includes all our dependency jars. Luckily, all of this can be accomplished in an automated way by a little bit of maven magic. All you have to do is to replace the Maven Shade plugin with two other plugins. The Dockerfile needs only one additional line.

This is how you do it:

At first, we will have to remove our maven-shade-plugin, that might look look this:

The maven-dependency-plugin will download all our dependency jars to our local directory /target/dependency-jars during the package phase.

The maven-jar-plugin will create our application jar as /target/application.jar . Additionally, it will create a manifest and add all jars from the directory /target/dependency-jars to our application classpath.

After executing …

$ mvn clean package

… we will have the following directory structure:

The contents of our target directory

The file application.jar contains a generated manifest that looks like this:

Starting our application with this command:

$ cd target
$ java -jar application.jar

… will work, as long as there is the directory dependency-jars alongside with all necessary dependencies. Maven will take care of that.

Dockerize it

The layout of our /target directory can be utilized during a docker image build.

Our Dockerfile copies all our dependencies and the application jar into the image. When starting the app, all dependencies will be available.

There is another benefit that we get from using Docker. See this paragraph from the Docker documentation concerning its build cache:

When building an image, Docker steps through the instructions in your Dockerfile, executing each in the order specified. As each instruction is examined, Docker looks for an existing image in its cache that it can reuse, rather than creating a new (duplicate) image.

So as long as the dependencies don’t change, the line

COPY target/dependency-jars /run/dependency-jars

will be executed by Docker using its build cache. This can be seen in the debug output:

[INFO] — -> Using cache
[INFO] — -> 964d6819c057
[INFO] Step 20/24 : COPY target/dependency-jars /run/dependency-jars

Conclusion

As you can see, only minor changes to your maven build and Dockerfile are needed to get rid of an overly long and inefficient build process.

You will take load off your CI servers and your software repositories will save storage space when they lo longer have to keep your fat jars.

A fully working example of all my code snippets can be found on Github.