Java Learning — Fat Jars, Dependency Shading, and upgrading from ant to gradle
So I’ve been working on the KBase NarrativeJobService Execution Engine. Aka, the backend infrastructure for KBASE for launching jobs. Before, we used a custom backend infrastructure, but I have been working on getting it to run on HTCondor instead. (The folks at HTCondor are super nice, and HTCondorWeek was great)
Anyway, I ran into a problem with upgrading the docker java libraries, as they had “transitive dependencies” on other libraries, such as Jackson, and those conflicted with the Jackson libraries already being used by the project. Luckily, there is something called Library Shading, in which dependencies are renamed and thus allowing you to use multiple versions of the same library. Thank you project-aries : https://github.com/project-aries/docker-java-shaded
Switching to gradle was suprisingly easy. We took custom jars and uploaded them into our Artifactory, and then installed them in the gradle file. Since our artifactory is configured to search JCenter, we can even just search Jcenter for our desired jar, and copy and paste the gradle import statement from there, and our own artifactory will download it for us.
Gradle also has a “shadow-jar” plugin, which I used to build a “Fat jar”, which is a jar that contains all of the dependencies for the project, instead of having many, many jars. This would have been the plugin I would have used to perform “package relocation” on the docker-java libraries and to be able to include them into the build.
I included the gradlewrapper in the github repo, so it automatically downloads the correct version of the gradle jar, which means the system in which I am running the build on doesn’t even need gradle installed. Pretty neat!