Open in app

Sign in

Write

Sign in

JPMS Support For Module Versions — A Research Log

CodeFX Weekly #54 — 26th of January 2018

Nicolai Parlog
nipafx news
Published in
10 min readJan 31, 2018

--

Hi everyone,

it’s Friday morning and I’m looking at a full schedule (including flying to Sofia for a presentation at JUG Bulgaria) and an empty list of ideas for today’s weekly. Until just now, when I had one. I wanted to use the time on the road to experiment with the little support for module versions that the JPMS has to offer. I usually don’t write “research logs”, but I have seen it recommended a couple of times, so why not try it out?

Here’s the deal: I’m gonna write down what I do while I do it and we’ll see where it gets us. Then tonight at the hotel, I’m gonna send it out after a little polish. But no backsies, I’ll just improve phrasing and put in a few headings.

This is new for me and I guess for you, too, so I’m really interested in your opinion. Did it work out? Did you like reading along or am I rambling? Let me know under nipa@codefx.org or on Twitter.

I send this newsletter out every Friday. Yes, as an actual email. Subscribe!

Recording module versions in JPMS

Setting up the experiments

[1450, I’m at the train station in Karlsruhe and have a few minutes before my train to Frankfurt airport arrives.]

Let’s start with --module-version on javac and jar. This is what the command line help has to say:

--module-version 
    Specify version of modules that are being compiled--module-version=VERSION
    The module version, when creating a modular
    jar, or updating a non-modular jar

Mmh, ok. Apparently reading time is over, time to experiment. To keep things simple and fast, I’m starting with the Hello Modules demo application. And I just now realize that there isn’t a build script for it, so let’s quickly write one:

#!/bin/bash
set -eecho "--- COMPILATION & PACKAGING ---"echo " > creating clean directories"
rm -rf target
mkdir -p target/classesecho " > compiling"
javac \
    -d target/classes \
    --module-version 0.1 \
    $(find src -name '*.java')echo " > packaging"
jar --create \
    --file target/hello-world.jar \
    --module-version 0.2 \
    --main-class org.codefx.demo.jpms.HelloModularWorld \
    -C target/classes .

Then, to see where the information ends up, I’ll prod Module - here's the demo's updated main method:

System.out.println("Hello, modular World!");Module module = HelloModularWorld.class.getModule();
ModuleDescriptor descriptor = module.getDescriptor();
System.out.println(descriptor.rawVersion());

[1510, the train was a little late, but it arrived eventually. This is the last time I can go online before the hotel in a few hours, so if there’s anything I need to look up, it better comes up now. Also, I just realized that I forgot my little talk tech bag with the presenter and HDMI adapter. Damn. Thankfully, a quick email exchange with Martin and Ivan from JUG Bulgaria fixes that — they have me covered. Thanks, guys!]

Building and running prints the following:

> Hello, modular World!
> Optional.empty

(You can see here why rawVersion returns an Optional.)

As expected. Note that I’ve so far done nothing but to set up the experiments (with the build script) and consolidate my knowledge (by looking at rawVersion before adding version information). This way, experiments are as quick as I can make them and have a clear reference point to compare observed behavior against.

The --module-version option

Now it’s time to actually use the new flags. First, I compile with --module-version 0.1, then run the thing. I'm not sure what exactly to expect - you'd think the version shows up at run time, but then what's jar --module-version there for? The same? My gut feeling says the compiler won't embed the version.

> Hello, modular World!
> Optional[0.1]

And I was wrong. So what does jar --module-version do? Let's use version 0.1 for javac and 0.2 for jar. I assume jar, being called after the compiler, overrides its version.

> Hello, modular World!
> Optional[0.2]

Indeed. To verify, lets compare the two module-info.class files. I don't want to (or even know how to) do something fancy, so let's see what cat classes/module-info.class says:

����5SourceFilemodule-info.javaModule
0.1
9.0.1
module-info org.codefx.demo.jpms_hello_world    java.baseorg/codefx/demo/jpms�

Eh… Well I can see 0.1 in there. (You'll also notice that I didn't yet update to 9.0.4.) What does unzip -q -c target/hello-world.jar module-info.class report?

����5
 module-infomodule-info.java
SourceFileModulePackagesorg/codefx/demo/jpmsModuleMainClass&org/codefx/demo/jpms/HelloModularWorld
Module org.codefx.demo.jpms_hello_world
0.2      java.base9.0.1�

And there’s 0.2. Clearly, jar rewrites module-info.class, which is not at all surprising because it also has to add the main class. On a higher level of abstraction, I can use java --describe-module:

$ java
    # yes, when nobody looks, I use `-p` instead of `--module-path`
    -p target/classes
    --describe-module org.codefx.demo.jpms_hello_world> org.codefx.demo.jpms_hello_world@0.1 file:.../target/classes/
> exports org.codefx.demo.jpms
> requires java.base$ java
    -p target/hello-world.jar
    --describe-module org.codefx.demo.jpms_hello_world> org.codefx.demo.jpms_hello_world@0.2 file:.../target/hello-world.jar
> exports org.codefx.demo.jpms
> requires java.base

There it is, java --describe-module reports 0.1 for the exploded JAR and 0.2 for the packaged one. That is also as expected.

And you can see that the module path, like the class path, actually accepts exploded JARs, i.e. a folder like target/classes, which contains the raw .class files:

$ tree target/classes/> target/classes/
> ├── module-info.class
> └── org
>     └── codefx
>         └── demo
>             └── jpms
>                 └── HelloModularWorld.class
>
> 4 directories, 2 files

So what have we established so far?

  • javac --module-version embeds the specified version in module-info.class, from where tools and reflection API can read it
  • jar --module-version does the same, possibly overriding existing version information

Recording dependency versions

The module descriptor generated by javac also contained the string 9.0.1, which is the Java version I built this on. But I suspect the concrete string comes from the base module's version because java.base is also mentioned there. Also, I thought I once saw an option to record dependency information.

Search in progress…

Nope, neither javac --help | grep -C 2 version nor the same for jar show anything else that's related to module versions. Mmh, weird.

So let’s see whether other module versions show up. For that I switch to Monitor, the demo application for The Java Module System. It consists of a handful of modules and one of them, monitor depends on quite a few other ones, so if dependency versions show up, it should be there.

First, I sprinkle --module-version 0.2 on all jar commands, then build and look at the module descriptor for monitor:

$ java -p mods --describe-module monitor> monitor@0.2 file:.../mods/monitor.jar
> requires monitor.statistics
> requires monitor.rest
> requires monitor.persistence
> requires monitor.observer.beta
> requires monitor.observer
> requires java.base mandated
> requires monitor.observer.alpha
> contains monitor$ jar --file mods/monitor.jar --describe-module# almost the same, still no dependency versions$ unzip -q -c mods/monitor.jar module-info.class> ����5
>      module-infomodule-info.java
> SourceFileModulePackagesmonitorModuleMainClass
> monitor/Main
> Module0.2monitor.statistics
> monitor.restmonitor.persistencemonitor.observer.betamonitor.observer java.base▒9.0.1monitor.observer.alpha
:

Mmh, no, other versions aren’t recorded. But once again 9.0.1 is awfully close to java.base - let's throw in two other platform modules and build again.

����5!
module-infomodule-info.java
SourceFileModulePackagesmonitorModuleMainClass
monitor/Main      Module0.2monitor.statistics
monitor.restmonitor.persistence
java.desktop9.0.java.xmlmonitor.observer.betamonitor.observer ava.basemonitor.observer.alpha�
F

No, still just the one 9.0. (now even the 1 went missing).

Fortunately, I’m still on the train, so I am online. Look what I found in the list of Jigsaw issues:

#VersionedDependences — Consider allowing specific version strings, or perhaps version constraints, as an optional element of requires clauses in module declarations. Failing that, consider allowing specific version strings, or perhaps version constraints, to be added to the dependences recorded in a compiled module descriptor; this would, e.g., allow a compiler or build system to record the versions of the modules against which a particular module was compiled, for use by other tools. In either case, if such version information is merely informative then it will still honor the version selection non-requirement; if such version information is interpreted by the module system then that requirement may come into question. [Cristiano Mariano, Stephane Epardaud]

Resolution When compiling a module that depends on some other modules, record the version strings of those modules, if available, in the resulting module descriptor. [Proposal, as amended per a suggestion by Rémi Forax]

From the proposal:

Extend module descriptors to allow the inclusion of version strings in the requires table of the Module attribute by introducing a u2 requires_version_index field after the requires_flags field of each entry of that table. The value of this new field is either zero, to indicate that no version string was recorded, or else the constant-pool index of a CONSTANT_Utf8_info structure whose value is the version string.

Aha! So the versions are recorded! And they only show up once because they all reference the same constant pool index.

[1650, there aren’t many places I know at Frankfurt airport where you can get food and electricity — I’m at one of them now and just took a break, eating something while reading Persepolis Rising.]

Now I really wonder, whether there isn’t a better way to look into the module descriptor. I’m offline now, so I can’t ask the Internet, which means I have to run on what I know — that’s not a lot. I heard javap makes bytecode presentable (does it decompile?), so let's use it:

$ javap classes/monitor/module-info.class
> Compiled from "module-info.java"
> module monitor {
>   requires java.base;
>   requires monitor.observer;
>   requires monitor.observer.alpha;
>   requires monitor.observer.beta;
>   requires monitor.statistics;
>   requires monitor.persistence;
>   requires monitor.rest;
>   requires java.xml;
>   requires java.desktop;
> }

Wow, that was shockingly easy. But it really just gives me back the source code (almost; note that it lists java.base, which is of course not in the original declaration). Can the options help?

Imagine elevator muzak, playing in the background

Nope, nothing. I always get the same output. Now I’m out of ideas for the descriptor. But this is not a dead end yet.

Accessing dependency versions

To continue quoting the proposal:

Extend the java.lang.module.ModuleDescriptor.Requires API with a single new method:

Optional compiledVersion();

This method will return an empty Optional object if no version string was recorded for the corresponding dependence, or else an Optional that contains the recorded version string.

I’ve actually seen this while working on the reflection chapter. I slip the following method into Monitor’s main module:

private static void outputVersions() {
    ModuleDescriptor descriptor = Main.class.getModule().getDescriptor();
    descriptor.requires().forEach(System.out::println);
}

Output:

> monitor.statistics (@0.2)
> monitor.rest (@0.2)
> monitor.persistence (@0.2)
> java.desktop (@9.0.1)
> java.xml (@9.0.1)
> monitor.observer.beta (@0.2)
> monitor.observer (@0.2)
> mandated java.base (@9.0.1)
> monitor.observer.alpha (@0.2)

Neat! To summarize:

  • the compiler records the versions of dependencies in the module descriptor
  • command tools like jar and java are hesitant to cough up the recorded versions
  • the reflection API, more precisely ModuleDescriptor::requires, make them available

So what is this good for? Good question. One immediate advantage is this:

> Exception in thread "main" java.lang.IllegalArgumentException
>     at monitor@0.2/monitor.Main.outputVersions(Main.java:46)
>     at monitor@0.2/monitor.Main.main(Main.java:24)

The stack trace contains the module version! Don’t say, nobody cares about that — at least I do, so the set of caring devs is not empty.

Hey, why not analyze recorded vs actual versions? I’m possibly running into the weeds here, but what about this?

public static void analyzeDependencyVersions(Module module) {
    module
        .getDescriptor()
        .requires().stream()
        .map(requires ->
            analyzeDependencyVersion(module, requires))
        .forEach(System.out::println);
}private static String analyzeDependencyVersion(
        Module module, Requires requires) {
    Optional compiledVersion = requires.rawCompiledVersion();
    String dependencyName = requires.name();
    String partialMessage = format(
        "%s -> %s: compiled @%s, ",
        module.getName(),
        dependencyName,
        compiledVersion.orElse("unrecorded"));    return module
        .getLayer()
        .findModule(dependencyName)
        .map(dependency -> {
            Optional actualVersion = dependency
                .getDescriptor()
                .rawVersion();
            String msg = partialMessage
                + "actual @"
                + actualVersion.orElse("unknown");
            if (!compiledVersion.isPresent()
                    || !compiledVersion.equals(actualVersion)) {
                msg += " <<< DANGER, DANGER!";
            }
            return msg;
        })
        .orElse(partialMessage + "");
}

Output:

> monitor -> monitor.statistics: compiled @0.2, actual @0.2
> monitor -> monitor.rest: compiled @0.2, actual @0.2
> monitor -> monitor.persistence: compiled @0.2, actual @0.2
> monitor -> java.desktop: compiled @9.0.1, actual @9.0.1
> monitor -> java.xml: compiled @9.0.1, actual @9.0.1
> monitor -> monitor.observer.beta: compiled @0.2, actual @0.2
> monitor -> monitor.observer: compiled @0.2, actual @0.2
> monitor -> java.base: compiled @9.0.1, actual @9.0.1
> monitor -> monitor.observer.alpha: compiled @0.2, actual @0.2

[1800, it was quite a walk from that bar to the my gate, but at least there weren’t any lines at pass control and security. Much to my chagrin I just found out that the plug at the bar that I specifically chose because it offered them was apparently not working, so now I’m almost out of battery life.]

That’s boring, but if I jar --update --module-version a couple of dependencies, it should get more interesting:

$ jar --update
    --file mods/monitor.observer.jar
    --module-version 0.5
> 'u' flag requires manifest, 'e' flag or input files to be specified!
> Try `jar --help' for more information.

What? But I don’t want to add a manifest, define a main class (e), or add files. WTF, jar?! Eh... ok, maybe add an empty manifest?

That worked. After a few more edits I get:

> monitor -> monitor.statistics: compiled @0.2, actual @0.8 <<< DANGER, DANGER!
> monitor -> monitor.rest: compiled @0.2, actual @0.2
> monitor -> monitor.persistence: compiled @0.2, actual @0.1 <<< DANGER, DANGER!
> monitor -> java.desktop: compiled @9.0.1, actual @9.0.1
> monitor -> java.xml: compiled @9.0.1, actual @9.0.1
> monitor -> monitor.observer.beta: compiled @0.2, actual @0.2
> monitor -> monitor.observer: compiled @0.2, actual @0.5 <<< DANGER, DANGER!
> monitor -> java.base: compiled @9.0.1, actual @9.0.1
> monitor -> monitor.observer.alpha: compiled @0.2, actual @0.2

Danger! Give it a little polish and this could make some informative log output.

With that I think that particular dead horse is sufficiently beaten. The summary of summaries:

  • javac --module-version embeds the specified version in module-info.class, from where tools and reflection API can read it
  • jar --module-version does the same, possibly overriding existing version information
  • the compiler also records the versions of dependencies in the module descriptor, from where the reflection API makes them available
  • one use case would be comparison of compiled against actual versions

Oh, wait. I also checked Maven and it didn’t include version information yet. That should be straightforward — I’ll open an issue over the weekend.

[1815, and with that it’s almost time to board. I wanted to go over jar --hash-modules, too, but instead I think I'm gonna wrap this up. I'll add in some links and do a little proof reading, so I can send it out as soon as I arrive at the hotel. Also, battery is running out...]

Shots

I assume all of you know what net neutrality means and I hope you are fighting your government if it is trying to take it away from you. Part of that is explaining to non-technically inclined people why it is important and, stupid as it might be, Burger King’s video on Whopper neutrality can actually do that in a fun way. So if you don’t mind being part of a fast food empire’s marketing scheme, share it with the burger lovers you know that need to learn about net neutrality.

so long … Nicolai

PS: Don’t forget to subscribe or recommend! :)

Photo by Kari Shea on Unsplash
Java
Java9

--

--

Nicolai Parlog
nipafx news

Written by Nicolai Parlog

Editor for

Nicolai is a #Java enthusiast with a passion for learning and sharing — in posts & books; in videos & streams; at conferences & in courses. https://nipafx.dev

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams