GeoTools On Java 9
CodeFX Weekly #31 — 18th of August 2017
A project I’m migrating to Java 9 (Cadenza, well summarized as “Google Maps meets Excel”) uses GeoTools, “an open source Java library that provides tools for geospatial data”. It seems to be a neat lib (haven’t really used it myself yet) and I spent some time with it because it found a very ingenious way to fail on Java 9. Let me tell you about that…
GeoTools is an open source Java library that provides tools for geospatial data.geotools.org
I send this newsletter out every Friday (well, this time it was Sunday). Yes, as an actual email. Subscribe!
Project of the week: GeoTools
Since we’re getting to know GeoTools anyway, I picked it as project of the week and decided to lead with that. Here are the project’s core features:
Definition of interfaces for key spatial concepts and data structures
- Integrated Geometry support provided by Java Topology Suite (JTS)
- Attribute and spatial filters using OGC Filter Encoding specification
A clean data access API supporting feature access, transaction support and locking between threads
- Access GIS data in many file formats and spatial databases
- Coordinate reference system and transformation support
- Work with an extensive range of map projections
- filter and analyze data in terms of spatial and non-spatial attributes
A stateless, low memory renderer, particularly useful in server-side environments.
- compose and display maps with complex styling
- vendor extensions for fine control of text labels and color blending
Powerful schema assisted parsing technology using XML Schema to bind to GML content. The parsing / encoding technology is provided with bindings for many OGC standards including GML, Filter, KML, SLD, and SE.
GeoTools Plugins: open plug-in system allowing you to teach the library additional formats
- Plug-ins for the ImageIO-EXT project allowing GeoTools to read additional raster formats from GDAL
GeoTools Extensions_: Provide additional capabilities built using the spatial facilities of the core library. Extensions provide graph and networking support (for finding the shortest path), validation, a web map server client, bindings for XML parsing and encoding and color brewer!
GeoTools Unsupported: GeoTools also operates as part of a wider community with a staging area used to foster new talent and promote experimentation. Some highlights are swing support (used in our tutorials!), swt, local and web process support, additional symbology, additional data formats, generation of grids and a couple of implementations of ISO Geometry.
Now off to my problem!
GeoTools on Java 9
You see, GeoTools has a plug-in system that allows users to extend the library. Dutifully eating its own dog food, it amply uses that API itself, and as a consequence GeoTools is useless without it. Now guess which part stopped working on Java 9…
The plugin-systems has a
FactoryRegistry at its core, which allows registering and requesting factories by their type. Furthermore, these factories are categorized by a common supertype. For example:
- the category
- the category
DateFactoryand the same
To express preferences (I think) it is possible to declare a partial order across factories without having to provide a proper
Comparator. So when iterating over all factories from a category, the iteration will respect the orderings that were defined between selected factories.
Instead of implementing all of that, GeoTools relied on the
ServiceRegistry class from the package
javax.imageio.spi and extended it with its
FactoryRegistry. That worked fine in Java 8 - not so much with Java 9.
Brazenly breaking bonds
This happens, when you run GeoTools on Java 9:
is not an ImageIO SPI class
Weird, what does
Err, ok? Why is this happening? Looking at the Java 9 documentation, you can find the comment:
The set of categories supported is limited to the following standard Image I/O service types:
An attempt to load a provider that is not a subtype of one of the above types will result in
No explanation why that decision was made, though. Turns out the change was implemented in JDK-8068749, which says:
lookupProviders methods in
javax.imageio.spi.ServiceRegistry are wrappers around
ServiceLoader.load. This is problematic for modules because the
ServiceLoader.load method looks at their caller to check if the caller is in a module and if so, that the caller's module has the right "uses" in its module declaration. For now, then calling
lookupProviders will cause
ServiceLoader to check that
ServiceRegistry's module (java.desktop) declares the "uses" and this will probably fail because the java.desktop module is unlikely to have it.
The alternative is to do nothing and just deprecate these methods. This option may not be too bad if we can establish that these methods are rarely, if ever, used outside of the JDK.
Long story short: It’s the module system’s fault. The result is that the
ServiceRegistry API was broken rather brutally and without any warning. I wouldn't blame GeoTools for running into that problem, but it is a problem now and needs to be fixed.
On the beaten path
I was not the first to run into that problem. In fact it was November 2015 (wow!) when this was first discovered and an issue was created. Just two days later Vincent Privat opened a pull request, which proposed a solution. So why are we still having this problem now, more than 18 months later?
The latest Tweets from Vincent Privat (@VincentPrivat). #Toulouse, #OpenData, #OpenStreetMap, #JOSM core dev. Toulousetwitter.com
Vincent’s initial implementation simply copied the JDK source code. That’s a pragmatic solution and good enough to get things going locally, but GeoTools couldn’t merge the PR for legal reasons: Open JDK code is licensed under GPL v2 with the class path exception, GeoTools under LGPL.
A new PR was opened but it had similar problems and then a slow discussion with some messages going back and forth every other month slogged through 2016. From the outside it looks like Vincent’s solution worked too well for him to rewrite his solution from the scratch and the GeoTools maintainers didn’t feel the pain.
I did, though, because GeoTools is one of three major blockades in our Java 9 migration and I wanted to clear it. So: fork, clone, branch, remove
extends ServiceRegistry, follow the compile errors and, et viola, a few hours later I had a GeoTools version that works perfectly on Java 9 and doesn't use any OpenJDK code. I opened a pull request, but I see first signs of the waiting game beginning again. Lets see how long it takes to polish (needs documentation) and merge it.
There’s not much to say regarding the implementation details. The partial ordering was interesting but 80% of the solution already existed, so I didn’t have to think too hard, just extend it a little but and clear a few bugs.
Oh yeah, one thing I did was deprecate all the methods that come from the
ServiceRegistry API because they talk so much about providers (e.g.
registerProvider) and create variants whose names sre more suited for a
registerFactors). I also started to replace the use of iterators with streams and the many legal
null returns with
Optional but that proved to be difficult in quite a few edge cases and I stopped to find out whether the maintainers are interested in that. Let's see...
- Bohemian Haskell
- looks like Bannon just left the White House — fuck yeah, one lunatic less