Dynamic Linkage to a z/OS Native Library in Java
To access core z/OS features from a Java mainframe application, one would need to consider the use of the Java Native Interface framework. A few examples of such need could be found in the interaction with the built-in z/OS Communication Server component named AT-TLS (see: AT-TLS for the Modern Mainframe) or the direct interaction with the z/OS security managers such as RACF, TopSecret, ACF/2, etc. This article will try to narrow down the minimum requirements on how to link with native libraries, how to package, deliver and deploy the resulting programs easily and concisely.
Note: all the code examples below can be found in this public repository.
Firstly, a collection of needed libraries is required. Putting them all in a folder, set their executable and program-controlled (z/OS only) flags, and add them to the library path environment variable. The following Unix commands will do the trick:
extattr +p *.so # only for z/OS
chmod a+x *.soThere are two ways to propagate the information about native libraries’ location to the Java runtime: one is via the system variable `LIBPATH`, and the other is via the application argument `-Djava.library.path`. Both serve the same purpose but the application argument has precedence over the environment variable. Below is a list of commonly used z/OS paths where other libraries (e.g. JZos, IRR, etc.) could be found. Those directories are often added to the application library path arguments or the environment variable:
- /lib
- /usr/lib
- ${JAVA_HOME}/bin
- ${JAVA_HOME}/lib/s390x
- ${JAVA_HOME}/lib/s390x/j9vm
- ${JAVA_HOME}/bin/classic
Note: The library paths could be different on a particular installation.
The next step is to load the library into memory, which could be done in two ways. The more straightforward way is to load the library by specifying the absolute path and exact filename using the method `java.lang.System#load`. The second and preferred way is to use the method `java.lang.System#loadLibrary` with the library name. The filename is then derived from the library name by adding the `lib` prefix and using the `.so` or `.dll` file extension, depending on the platform.
// using absolute path
static {
System.load("/z/user/libfeature.so");
}
// using library name
static {
System.loadLibrary("feature");
}This, unfortunately, is not everything needed when dealing with dynamically loaded libraries. It may be enough during the development phase but there are more details needed before a complete release of the software can be completed. The first consideration is to determine the platforms on which the application would be expected to run. More frequently, multiple platforms will be supported: z/OS, Linux, Unix, 64-bit, 32-bit, 31-bit (z/OS), etc.
To allow the application to run on multiple platforms off z/OS it is necessary to create two implementations with the same interface and a factory method (or a similar approach). The decision of which implementation to use should be based on the system property `os.name`. In z/OS it returns the value “z/OS’’.
interface NewFeature {
void doStuff();
}
class NewFeatureZos implements NewFeature {
@Override
public native void doStuff();
}
class NewFeatureDummy implements NewFeature {
@Override
public void doStuff() {
System.out.println("Platform not supported.");
}
}
@lombok.experimental.UtilityClass
class Factory {
public NewFeature getNewFeature() {
if ("z/OS".equals(System.getProperty("os.name"))) {
return new NewFeatureZos();
} else {
return new NewFeatureDummy();
}
}
}The packaging of the application should contain native binaries for each combination of platform and operating system that is planned to be supported. During runtime, the application would need to decide which library file to load based on system properties, such as the `com.ibm.vm.bitmode` available on z/OS, which returns “32” for 31-bit or “64” for 64-bit architecture. Other JDKs use another property named `sun.arch.data.model`. An updated implementation of the factory class may look something like this:
class NewFeatureZos implements NewFeature {
static {
if ("32".equals(System.getProperty("com.ibm.vm.bitmode"))) {
System.load("feature-31");
} else{
System.load("feature-64");
}
}
@Override
public native void doStuff();
}Note: The application packaging must include both files `libfeature-31.so` and `libfeature-64.so`.
With the recent announcement of JDK 11 on z/OS (aka IBM Semeru Runtime Certified Edition, Version 11), comes the possibility of loading a 31-bit library under 64-bit JDK, which makes it possible to release an application with fewer versions of its native dependencies. Nevertheless, delivering binaries for each architecture is still preferred mainly for performance reasons but also for backward compatibility with JDK 8.
Besides packaging, the application’s life cycle includes release and deployment. From a native library point of view, the binary files should be included in a PAX (or some of its alternatives) and an installation script should contain the logic for deploying, modifying, and running the program with proper configuration (e.g. library paths). It is more difficult when the libraries are provided by a 3rd party in a bundled JAR file, in which case the JAR would need to be extracted before including them in a release. A change in any of the native dependencies of a Java application would require repeating the whole release process.
An experienced developer would always consider other options of delivering their application to its users, more concretely for a java application, packaging the program in a JAR file could always be an alternative. The question here would be if packaging the native libraries and loading them on request would be possible. If it is, then bundling the native libraries will definitely save developers time in the release process. There is one specifying thing in z/OS to do before and it is that native libraries should be program-controlled on the platform. To set this flag, the application user requires `READ` access to the `BPX.FILEATTR.PROGCTL` resource in the `FACILITY` class.
@lombok.experimental.UtilityClass
class NativeLibraryUtils {
/**
* Collection of a loaded file to avoid loading
* the same library multiple times
*/
private final Set<String> loaded = new HashSet<>();
private File extract(String filename) {
// create a copy of library in the temporary folder
File libFile;
try (InputStream is =
NativeLibraryUtils.class.getResourceAsStream(filename)
) {
libFile = File.createTempFile(filename, ".so");
try (OutputStream os = new FileOutputStream(libFile)) {
IOUtils.copy(is, os);
}
} catch (IOException ioe) {
throw new IllegalStateException(
"Could not load native library `" + filename + "`",
ioe
);
}
// change file to be executable
libFile.setExecutable(true);
// change file to be program-controlled
// this step is required only for z/OS
FileAttribute.setProgramControlled(
libFile.getAbsolutePath(), true
);
// clean up file after application will be stopped
libFile.deleteOnExit();
return libFile;
}
private void load(String fileName) {
// check if the file has not been loaded yet, if not load it
synchronized (NativeLibraryUtils.class) {
if (!loaded.contains(fileName)) {
File file = extract(fileName);
System.load(file.getAbsolutePath());
loaded.add(fileName);
}
}
}
public void loadLibrary(String libraryName) {
// construct name of file
StringBuilder sb = new StringBuilder();
sb.append("/lib/").append(libraryName);
if ("32".equals(System.getProperty("com.ibm.vm.bitmode"))) {
sb.append("-31");
} else {
sb.append("-64");
}
sb.append(".so");
// load the library
load(sb.toString());
}
}In the example above it can be seen that the class of libraries functions also dynamically detects if they have been loaded or not. This permits multiple instances of this class to be used in different parts of the Java application without conflicts. The class that implements the native methods’ interface can now be simplified to just this:
class NewFeatureZos implements NewFeature {
static {
NativeLibraryUtils.loadLibrary("feature");
}
@Override
public native void doStuff();
}At this moment everything is handled in the utils implementation and the handling of the native libraries remains behind the scenes. The library path is no longer necessary either since it is only used to include the binaries in the JAR file, package, and release it — simple and clean.
Co-authored by Boris Petkov
