Auto-conversion in z/OS UNIX
🖥 🖥 🖥 Open source software plays a significant role in application modernization. On an EBCDIC platform such as z/OS, porting open source software is harder than porting it to ASCII platforms. A common porting problem is the ASCII bias in the code that deals with text data, because most programmers use ASCII encoding properties. ASCII encoding is different from EBCDIC and a number of typical properties such as 'a'<= var <='z' when checking for lowercase alphabetic characters, do not work for EBCDIC.
Finding all cases where programmers used ASCII properties and changing them to work for EBCDIC is hard and error prone and each time a new version of the application is released the same process has to occur again and again. For this reason, a more pragmatic approach is to run such applications as ASCII applications on z/OS. The side-effect of this approach is that the input and output has to be in ASCII.
Most data on z/OS is in EBCDIC and cannot be read by ASCII applications without being converted to ASCII. Output from ASCII applications to a terminal is ASCII and shows up garbled unless converted to EBCDIC. A solution to this problem is to use auto-conversion to enable ASCII applications to read input and to produce output in EBCDIC.
If you are not familiar with auto-conversion, you can read this article by Leonard Carcaramo Jr.
To use auto-conversion on z/OS you have to enable it before invoking an ASCII application. This can be done in your local shell environment or globally on the system level. This approach is not without undesirable side effects and the documentation advises enabling auto-conversion per application and not globally.
In this article you will discover that auto-conversion can be set from within the application so it can run correctly even if your shell environment does not enable auto-conversion.
How to enable auto-conversion from within the application
To run an ASCII application one has to enable auto-conversion in the shell environment in order to see readable output from the application.
Enabling auto-conversion from within an ASCII application can be done using #pragma runopts or using Language Environment (LE) exit that allows setting LE environment from within the application.
Using an LE exit is a more generic approach as it works with C/C++ compilers that do not support #pragma runopts .
The following is an example of building a simple ASCII application that uses this approach.
Building example application for 32-bit and 64-bit is just a matter of compiling the same source with different compiler options, but 32-bit and 64-bit LE exits are different files and need to be paired with the corresponding build of the application. In either case the application does not have any sensitivity to the LE exit. It can be compiled and linked without the LE exit, in which case it will not have auto-conversion enabled and will produce garbled output in a shell environment in which auto-conversion is disabled.
The example below is a simple “Hello World” application that checks whether auto-conversion is active and displays the result in a message.
Standard streams in z/OS UNIX are implicitly tagged with IBM-1047 EBCDIC code page. This enables auto-conversion to compare the code page associated with standard streams and the code page of the application and if they are different perform the right code page conversion. As a result, the ASCII application in this example produces readable output to the terminal even if it is run in a shell environment in which auto-conversion is disabled.
By default files created in a shell environment with auto-conversion disabled are untagged. Even when enabled from within the application, the lack of file tag will prevent auto-conversion from converting input or output when it is redirected to a file. For this reason, when the input or output is redirected the file must be tagged with a desired code page. Assuming the output is redirected into a file namedout, the following shows how to tag the file:
> touch out
> chtag -tc 819 out # for ASCII output
> chtag -tc 1047 out # for EBCDIC output
> ls -T out
t IBM-1047 T=on outThe same approach works for both input and output files when they are redirected. In the case of input file the code page must reflect the actual code page of the file. In the case of output file the code page is driven by what encoding is needed in that output file and could be either ASCII or EBCDIC.
The content of the example application
The example application is checking the state of auto-conversion and prints out the results. This is not required for the application to run correctly, it only serves as a confirmation that the auto-conversion is enabled. Without LE exit the application would produce garbled output in a shell environment where auto-conversion is disabled. The example is a single source application, but the effect of LE exit applies to applications of any complexity. The example is intended for execution on z/OS and assumes you have access to bash, gmake, as and xlc.
LE exit for 32-bit applications is written in HLASM assembler and it defines all environment related variables that are required to enable auto-conversion.
LE exit for 64-bit applications is also written in HLASM assembler and it defines all environment related variables required to enable auto-conversion.
The build.sh shell script is used to invoke the makefilethat builds and runs both examples and does the cleanup of the build directory. Without parameters, it invokes the makefileto build and run both examples, if a clean parameter is specified, it will invoke the makefile to cleanup all output files produced during the build process.
The shell script is designed so it can be invoked from a build directory of your choice where all build outputs will reside. This allows building and running the example outside the directory where the example code resides. The idea is to use two separate shell sessions, one in which auto-conversion is enabled and one in which it is disabled and then try running the example applications from either session to confirm that the example applications produce readable output in both sessions.
The makefile is a GNU make that performs the build and run for both examples. The GNU make is not part of z/OS so if you do not have it you need to install a Rocket Software port of GNU make. Once both applications are built, running the make file repeatedly will only run the example applications. The makefile has the target clean that removes all output files created by the build process.
Closing remarks
Enabling auto-conversion from within the application can simplify porting open source software and allow it to be used even in shell environments where auto-conversion is not desirable. Tagging text files that are used by a ported software is a small price to pay for the benefit obtained from using the software.
All code mentioned in this article is available here
