AT-TLS Aware Mode in a Mainframe Java Application

In the previous two articles of this series, AT-TLS for the Modern Mainframe Java Applications and AT-TLS Configuration for Mainframe Java Applications the story of AT-TLS for Mainframe Java applications took shape. Going through the introduction and now that a server configuration is available, it is time to present what runtime handling is needed for the proper implementation of the AT-TLS Aware Mode by the modern mainframer.

The use of AT-TLS aware mode requires native code to interact with the core z/OS services. Luckily, it can be found readily available in this Broadcom source code contribution to the Zowe project. The native code should focus on a few aspects: to effectively connect sessions with the client’s code and synchronize multiple calls for details during a single request.

Requests use sockets that are identified by a number (ID), which matches their corresponding file descriptor in the operating system. This ID is used to communicate with the IOCTL service responsible for fetching information about AT-TLS states per each socket. The ID remains hidden inside the web server, e.g. Tomcat, and it usually remains inaccessible from Java code.

A simple strategy to avoid multiple system calls is to ensure those calls are made only once. The essential information necessary, in this case, would be the current state of the channel (secure vs. insecure) and the user ID (in the case of client certificate authentication). From the perspective of the native code library, a Data Transfer Object (DTO) could be used to collect the information once and then store it for future reference. The Zowe AT-TLS library solves the problem by storing IDs and binary data inside a class named `AttlsContext`, accessible via `InboundAttls`. It uses `ThreadLocal` to map the right AT-TLS context with the calling thread.

Below is a minimalistic example of a java wrapper class that uses ThreadLocal (note that the native library is loaded in a static initialization block the first time the application uses the class):

package com.mycompany;

import lombok.Getter;

public class Attls {

    private static final ThreadLocal<AttlsInfo> threadToAttls = new ThreadLocal<>();

    static {
        System.loadLibrary("attls");
    }

    public static void init(int socketId) {
        threadToAttls.set(new AttlsInfo(socketId));
    }

    public static AttlsInfo get() {
        return threadToAttls.get();
    }

    public static void dispose() {
        threadToAttls.remove();
    }

    @Getter
    public static class AttlsInfo {

        private final int socketId;
        private boolean secured;
        private String userId;

        AttlsInfo(int socketId) {
            this.socketId = socketId;
            fetchData(socketId);
        }

        private native void fetchData(int socketId);

    }

}

The base class AttlsInfo is instantiated with a socket ID parameter for each request and during application initialization. It is responsible for calling the native function `fetchData`, which would call the IOCTL service and would fetch two things: secure connection status and user ID (in case the request is signed by a client certificate). This wrapper class would be calling native code. To generate the interface (header file) you can use the following command: `$JAVA_HOME/bin/javah -jni com.mycompany.Attls`. The resulting code is then saved in attls.h. The method definition could be copied into the attls.c file. That is the required step before writing native code.

The final implementation may require some adjustments as shown in the example below:

#include "attls.h"

#include <resolv.h>
#include <ezbztlsc.h>

// this pragma generates string value encoded in ASCII
#if defined(__IBMC__) || defined(__IBMCPP__)
#pragma convert(819)
#endif

const char *ILLEGAL_STATE_EXCEPTION = "java/lang/IllegalStateException";
const char *SECURED = "secured";
const char *USER_ID = "userId";
const char *SIGNATURE_STRING = "Ljava/lang/String;";
const char *SIGNATURE_BOOLEAN = "Z";

#if defined(__IBMC__) || defined(__IBMCPP__)
#pragma convert(0)
#endif

JNIEXPORT void JNICALL Java_com_mycompany_Attls_00024AttlsInfo_fetchData (JNIEnv *env, jobject obj, jint socketId)
{
   struct TTLS_IOCTL ioc = {0};

   // fulfill request
   ioc.TTLSi_Ver = TTLS_VERSION1;
   ioc.TTLSi_Req_Type = TTLS_QUERY_ONLY;

   // call ioctl
   int rc = ioctl(socketId, SIOCTTLSCTL, (char*) &ioc);

   // if ioctl returns an error throw exception
   if (rc < 0) {
       char message[256] = {0};
       snprintf(message, 255, "Cannot obtain AT-TLS data about socket ID=%. Return code: %d", socketId, rc);
     __etoa(message);
       jclass exception = (*env) -> FindClass(env, ILLEGAL_STATE_EXCEPTION);
       (*env) -> ThrowNew(env, exception, message);
       return;
   }

   // get class of AttlsInfo
   jclass clazz = (*env) -> GetObjectClass(env, obj);

   // check if connection is secured and store the flag in AttlsInfo
   // 1 = non-secure, 2 = handshake in progress, 3 = secure
   jfieldID secured = (*env) -> GetFieldID(env, clazz, SECURED, SIGNATURE_BOOLEAN);
   (*env) -> SetBooleanField(env, obj, secured, (jboolean) (ioc.TTLSi_Stat_Conn == 3));

   // get userId, convert to ASCII and set into AttlsInfo
   jfieldID userId = (*env) -> GetFieldID(env, clazz, USER_ID, SIGNATURE_STRING);
   __etoa(ioc.TTLSi_UserID);
   (*env) -> SetObjectField(env, obj, userId, (*env) -> NewStringUTF(env, ioc.TTLSi_UserID));
}

This sample code is quite self-explanatory. It starts with declaring the required text literals so they are compatible with JNI, which mandates the use of ASCII encoding. The code continues with the initialization of the IOCTL structure, in the concrete example, it resets all fields and sets the AT-TLS version and request types. After the IOCTL call is executed and no error is detected, the program would fetch the data and store it in the `AttlsInfo` class transcoding the EBCDIC strings to ASCII encoding in the process.

Building and linking the native code library can be done with the following commands:

xlc -W "c,langlvl(extended),dll,xplink,exportall" -qsearch=$JAVA_HOME/include -qsource -g  -c -qlist=libattls.cpp.lst -o attls.o attls.c
xlc++ -W "l,lp64,dll,dynam=dll,xplink,map,list" -g -qsource -o libattls.so $JAVA_HOME/bin/j9vm/libjvm.x attls.o > libattls.bind_64.lst

The produced binary file would need to have its accessors changed to be executable and program-controlled:

extattr +p libattls.co
chmod a+x libattls.so

Finally, the file location should be added to LIBPATH. Note that in Java, the library is loaded just by the name “attls”; the prefix “lib” and suffix “.so” are added automatically by the JVM (see. Dynamic Linkage to a z/OS Native Library in Java).

The entire functionality of AT-TLS revolves around a socket ID; it is required to obtain this value at the beginning of each request. The use of an embedded Tomcat server is the one used in the example below but the principle is the same for any other web server.

@Bean
public<T> TomcatConnectorCustomizer attlsOnEmbededTomcat() {
    return connector -> {
        try{
            ProtocolHandler protocolHandler=connector.getProtocolHandler();
    
            Field handlerField=AbstractProtocol.class.getDeclaredField("handler");
            handlerField.setAccessible(true);
            AbstractEndpoint.Handler<T> handler=(AbstractEndpoint.Handler<T>) handlerField.get(protocolHandler);
    
            handler=new AttlsHandler<>(handler);
    
            Method methodHandler=AbstractProtocol.class.getDeclaredMethod("getEndpoint");
            methodHandler.setAccessible(true);
            AbstractEndpoint<T,?> endpoint=(AbstractEndpoint<T, ?>) methodHandler.invoke(protocolHandler);
            endpoint.setHandler(handler);
    
            handlerField.set(protocolHandler, handler);
        } catch(Exception e) {
            throw new IllegalStateException("Cannot prepare AT-TLS handler",e);
        }
    };
}

This sample code obtains the original handler, customizes it as a delegator, and then sets it back to be used with the same interface and be able to recall all methods of the original instance. The next step would be to obtain the socket ID, initialize the AT-TLS object capable of fetching all necessary data, call the original method and then clean up to dispose of the AT-TLS state.

@RequiredArgsConstructor
class AttlsHandler<T> implements AbstractEndpoint.Handler<T> {

    @Delegate(excludes = Proccess.class)
    private final AbstractEndpoint.Handler<T> original;

    private Field fdField;
    /**
     * Methods take file description from all known sockets
     * @param socket implementation of socket in Tomcat
     * @return file descriptor value
     */
    private int getFd(SocketChannel socket) {
        if (fdField == null) {
            try {
                fdField = socket.getClass().getDeclaredField("fdVal");
                fdField.setAccessible(true);
            } catch (IllegalArgumentException | IllegalStateException | NoSuchFieldException e) {
                throw new IllegalStateException("Cannot find file descriptor (socket ID)", e);
            }
        }
        try {
            return fdField.getInt(socket);
        } catch (IllegalAccessException e) {
            throw new IllegalStateException("Cannot obtain file descriptor (socket ID)", e);
        }
    }

    /**
     * This method handles processing of each request. At first create AT-TLS context, the process and on the end
     * dispose the context.
     * @param socketWrapperBase describing socket (see Tomcat implementation)
     * @param status status of socket (see Tomcat implementation)
     * @return new status of socket (see Tomcat implementation)
     */
    public SocketState process(SocketWrapperBase socketWrapperBase, SocketEvent status) {
        NioChannel nioChannel = (NioChannel) socketWrapperBase.getSocket();
        Attls.init(getFd(nioChannel.getIOChannel()));
        try {
            return original.process(socketWrapperBase, status);
        } finally {
            Attls.dispose();
        }
    }

    interface Proccess {

        <T> SocketState process(SocketWrapperBase<T> socket, SocketEvent status);

    }

}

The difference between this custom implementation and the use of the Zowe library is about the InboundAttls class instance of Attls (as shown in the previous paragraph).

InboundAttls.init(getFd(nioChannel.getIOChannel()));
try {
    return original.process(socketWrapperBase, status);
} finally {
    InboundAttls.dispose();
}

In both cases, the result is an object accessible from any place in the code (except for new threads), capable of retrieving information about the AT-TLS state (`Attls.get()`, `InboundAttls.get()`, etc.).

The most important part of this implementation is the verification if the connection is secured or not. This is now possible to do through a servlet filter that calls `Attls.get().isSecured()` or `InboundAttls.getStatConn() == SECURE` for the Zowe library. If the connection is not secured, the processing should be interrupted as soon as possible (throwing an exception should be enough).

Co-authored by Boris Petkov

Attachment: The source code

Note: All AT-TLS constants could be found in IBM documentation.