Minimize Dependency Pain in Java Libraries

The Google Cloud Platform publishes client libraries for dozens of products from Cloud Asset to Cloud Web Security. These sit on top of much common infrastructure including Guava, gRPC, and protobufs as well as numerous third party open source projects like Netty and Jaxen. GCP adjacent projects such as Apache Beam and Spring Cloud GCP expand the scope and complexity even further.

Different teams and hundreds of developers in multiple time zones build these libraries. Ensuring that the latest versions of the libraries we publish all work together without conflicts is cat herding at epic scale.

Through the lessons we learned from developing and maintaining our own libraries for our customers, we have developed a collection of best practices for avoiding incompatibilities in Java libraries based on Maven Bill of Materials (BOMs) and semantic versioning. These practices are relevant to anyone shipping libraries that other Java projects depend on. We are now publishing these Google Best Practices for Java Libraries:

• JLBP-1: Minimize dependencies
• JLBP-2: Minimize API surface
• JLBP-3: Use semantic versioning
• JLBP-4: Avoid dependencies on unstable libraries and features
• JLBP-5: Avoid dependencies that overlap classes with other dependencies
• JLBP-6: Rename artifacts and packages together
• JLBP-7: Make breaking transitions easy
• JLBP-8: Advance widely used functionality to a stable version
• JLBP-9: Support the minimum Java version of your consumers
• JLBP-10: Maintain API stability as long as needed for consumers
• JLBP-11: Stay up to date with compatible dependencies
• JLBP-12: Make level of support and API stability clear
• JLBP-13: Quickly remove references to deprecated features in dependencies
• JLBP-14: Specify a single, overridable version of each dependency
• JLBP-15: Produce a BOM for multi-module projects
• JLBP-16: Ensure upper version alignment of dependencies for consumers
• JLBP-17: Coordinate rollout of breaking changes
• JLBP-18: Only shade dependencies as a last resort
• JLBP-19: Place each package in only one module
• JLBP-20: Give each jar a module name

These twenty practices describe how to organize a library and its own dependencies so developers can easily adopt them in their own projects without conflicts and linkage errors. I’ll summarize three of the most important practices here:

Minimize Dependencies (JLBP-1)

Each library should add as little to its dependents’ classpaths as reasonably possible, ideally nothing but itself. Adding a dependency for something that is difficult and complicated may be an acceptable tradeoff, but avoid adding a dependency merely to save a few lines of code. Every dependency of a library is also a dependency of the library’s consumers and increases the risks of version conflicts and security holes.

The other 19 best practices cover techniques for mitigating the impact of excess dependencies, but nothing is as effective as removing a dependency completely.

Publish and Consume BOMs (JLBP-15)

Sometimes a large dependency tree is unavoidable; for instance, when writing a Kubernetes application that integrates multiple GCP products like BigQuery, Cloud Datastore, Cloud Translate, and TensorFlow. Your project needs client libraries for each of those and all of their dependencies as well.

Rather than trying to pick and choose the versions of the dozens of artifacts such a project depends on, import a BOM and let it pick the versions for you. For example, com.google.cloud:libraries-bom guarantees consistent versions of all the artifacts in the Google Cloud Java orbit that work together and don’t conflict with each other. BOMs make updating to newer versions simpler since you only have to update one artifact instead of dozens of independent and potentially conflicting artifacts.

When you publish your own complicated, multi-module library, also publish a BOM and advise your customers to import it. If the product depends critically on other libraries, you might want to include those in your BOM as well. Make it easy for dependents to have consistent versions of everything in their classpath.

Semantic Versioning is Your Friend (JLBP-3)

Try really, really hard to avoid introducing incompatibilities once you ship (JLBP-8, JLBP-10, and JLBP-12) but when there’s simply no other choice, increment the major version. Bump the minor version when you introduce a new feature. Bump the patch version for everything else. The Semver spec lays out the details.

The breadth and depth of third party libraries available for developers to choose from is one of the key strengths of the Java ecosystem. However, too many choices present problems of their own. Carefully consider which and how many libraries you depend on. When you publish a library, take care not to add to your consumers’ dependency problems. Learning and following these 20 best practices lets you offer your users a smooth and problem free path to adopt your library.