Migrating to Java 11 while maintaining a Java 8 client library
The Auth-Service, Uptake’s Identity and Access Management suite, has been on Java 8 since the first git commit in November 2016. We have been big adopters of Java 8 features, including lambda expressions, Stream API, method referencing, optionals, and the Nashorn Javascript Engine. We use Nashorn to allow other teams to compose authorization policies in Javascript which are then compiled to Java.
In 2017, Java increased the frequency of their release schedule from two years to six months. However, only every third release receives long-term support. Java 9 and 10 are no longer supported. Java 11 is the first long-term supported version after Java 8.
Benefits of Java 11
While we were happy developing in Java 8, we noticed at least one deficiency and few new features we wanted. The configurability of Java running in our dockerized Kubernetes environments was limited. The JVM was unaware that it was running in a Docker container, and we relied on experimental compiler arguments to manage memory. Java 11 adds container support and more fine-grained control of docker memory.
We care deeply about code-quality. When we read that ErrorProne caught a bug in Java’s ConcurrentHashMap, we felt it might catch mistakes in our code too :). As a colleague says,
“We don’t always like static analyzers, but when we do, they’re developed by Google.”
ErrorProne has since caught missing @Overrides and @Test, and we’re still digging through the results. ErrorProne is not recommended for use in Java 8.
Mostly importantly to some team members, we were excited about some language features introduced in Java 11, such as List.of(). There are nearly 400 references in the codebase to Collections.singletonList(). It shouldn’t require so many characters to create a list of one element! Similarly, we’re excited about Optional.orElseThrow() instead of Optional.get(), and the new HTTP Client API.
Lastly, all developers have FOMO and want to use the latest and greatest thing. Most libraries get better with time (with a notable exception being Swagger-ui 2.x to 3.x). Our client library has continually evolved and improved. What new features were we missing in Java 11?
The Migration
Due to Java’s famous backwards compatibility, most of upgrading from Java 8 to Java 11 is simple blocking and tackling. We installed JDK 11.0.2, updated our docker base image, bumped Gradle and some of the libraries we rely on to their latest versions, and explicitly added a library which was part of JDK 8 but had been modularized in JDK 9. There were a few unit and integration tests that needing fixing due to a Nashorn Javascript bugfix and due to Java 9 Instant now having microsecond resolution (JDK-8068730).
As soon we had the application compiling with Java 11, we felt pretty jazzed! One of our highly-respected team members had never seen a migration in action — he had started with Java 1.2 and had moved to projects with later and later versions of Java. We were all excited about the migration and how the change would improve the codebase.
The Problem
As soon as we announced our migration plan, however, we were asked,
“What does this mean for the auth-client library? Will you continue to build a Java 8-compatible version?”
We realized a major problem: dozens of teams rely on our Java 8 client!
Mike Gorman, Chief Architect of the Uptake Platform, has written about when it’s okay to make breaking changes. His answer is that it’s never okay to break a customer, but that it’s okay to break internal teams if it’s well-communicated and the changes are limited in scope. However, we didn’t feel forcing teams to migrate from Java 8 to Java 11 to receive an updated auth-client was “limited in scope”.
Once we determined that maintaining a Java 8-compatible version was a requirement, our first hope was that compiled code written in Java 11 could be compatible in Java 8. Java supports backwards compatibility (ie., using Java 8 code while running Java 11), but unfortunately we found that the inverse is not supported: Upon setting sourceCompatibility 1.8 targetCompatibility = 1.8, gradle build publish would succeed, but the Java 11 code would fail once run in Java 8. Thus, we learned that specifying the sourceCompatibility and targetCompatibility was not sufficient (we had rediscovered JDK-8058150):
java.lang.NoSuchMethodError: java.util.List.of(Ljava/lang/Object;)Ljava/util/List;at com.uptake.auth.dto.User.setPrimaryEmail(User.java:143)
The Solution
We read a Gradle blog post that discussed how JDK 9 includes a new option — release <version> which cross-compiles to an older Java version. We added to our Gradle build file this code snippet and run gradle build -PcrossCompile during our CI/CD process.
Java 11 syntax in the client is now caught during the build process, ensuring bad code never gets published!
$ gradle build -PcrossCompile
> Task :client:compileJava FAILEDUser.java:143: error: cannot find symbol
emails = List.of(Email.buildPrimaryVerifiedEmail(id, email));
^
symbol: method of(Email)
location: interface List
1 error
FAILURE: Build failed with an exception.
The Auth-Service codebase has almost 60,000 lines of code. Almost 85% of it can use Java 11 features (while client src/main must stay on Java 8).
That makes us happy! And we can guarantee we won’t break customers or internal teams, keeping Mike Gorman happy.
