Hello World on z/OS
Let me tell you … karma is a bitch.
When I was in graduate school one of my required courses was an advanced statistics class that I genuinely loved but included a mandatory computer lab component the bored the shit out of me. It was a one and a half hour block every other week on how to use SPSS and most of the time it felt like at least 40 mins of that block was spent explaining how to click File -> Open. As a result I picked a computer where the screen was facing away from the instructor and spent most of the class time playing around on the University’s network (most famously figuring out how to patch into MSG’s live feed to watch the Rangers game without commercials)
Anyway, at the time I was completely unsympathetic to classmates that struggled to understand how to do something as basic as interact with their operating system’s interface. And then roughly eight years later I decided to write a blog post on the difference between transpiling COBOL and compiling it which required me to install and interact with IBM’s z/OS operating system…
Like I said: karma’s a bitch.
If you’ve followed any one of the amazing tutorials on how to set up a mainframe on a conventional personal computer, you’ve probably noticed they end with the login screen as if everything beyond that point will be intuitive and self-explanatory to newbies. I mean … that was my assumption going into this project. I’ll figure it out. How hard could it be? Maybe it would take me a few hours. Maybe I’d have to Google some stuff… read some documentation…
It took me over a week.
Over a week to figure out enough to compile and run a basic program.
It took me that long because coming from modern day software engineering environments to z/OS requires a herd of yak shaving. z/OS feels like a command line but is a text based interface that wants you to interact with it like a graphics based one, so you need to learn not just what the commands are but WHERE on the screen each command can be entered. Virtually everything you want to do requires you to write a JCL script and JCL on first glance is a gobbledygook of all caps. Default settings and library locations can be wildly different depending on the installation, with the stock answer on how to figure out your local ground truth being “call your sysadmin”. IBM has so much documentation and that documentation is so dense that it feels impossible for even Google to penetrate.
And then on top of all of that, IBM uses special and completely unintuitive names for basic concepts…. because OF COURSE THEY DO.
This drove me completely insane and my understanding of how things work and what to do is still very much a work-in-progress … but I did learn a lot and I’m happy for the opportunity to share so that I can do my small part to lower Google’s ad revenue from all those search queries you would otherwise be making.
Yak 00: Data sets are files! (Navigating z/OS)
Let’s say you’re starting like me. You have z/OS installed and the first thing you want to do is a hello world in COBOL or something. You figure there are probably sensible ways of moving the file from your host filesystem into z/OS but the program isn’t very long so to speed things along you’re just going to retype it. That sounds pretty straight forward right? Create a file, retype your program, save file, run compiler, execute program.
Except if you are like me you will quickly realize there appears to be no way to create files. Not on the menu, selecting the edit option immediately dumps you on this confusing screen asking for your data set’s location in the catalog, Googling “How do I create a file in z/OS” returns a bunch of info on z/OS Unix (which we will not be talking about here) and a bunch of stuff about creating data sets which isn’t relevant because you don’t want to create a data set you want to create a file!
Well … Data sets are files.
There I just saved you a lot of time and aggravation. What IBM calls “data sets” virtually every other OS on the planet calls files and directories. Data sets come in two flavors: sequential data sets which behave very much like files and partitioned data sets which behave more like directories on a traditional OS.
So to create a file you need to allocate a data set. For the purposes of programming you will probably want to create a partitioned data set (more on how to do that in a second) which will have a very specific naming convention you need to follow:
PROJECT.GROUP.TYPE(MEMBER)
Here’s what this all means:
PROJECT: Ridiculously unintuitive: this must be a userid. In theory the userid of the account you logged into TSO with. If you’re following along using hercules you’re probably usingibmuserto start out.GROUP: Think of this as your folder name. It can be anything you want as long as you stay within data set name limitations. Be mindful of both character length and moving too far beyond alphanumeric inputs.TYPE: What the file actually is. For our COBOL program this will be COBOL. For JCL it will be JCL. But as we move through the compiling, linking and executing phases there will be other less straight forward types z/OS will create for us like LOAD and OBJMEMBER: In our ‘data sets are files’ analogy this is your file name. LikeGROUPit can be pretty much whatever you want it to be with the same limitations. If you keep things straight forward and alphanumeric for the first program, you shouldn’t have to know too much about what z/OS will and will not allow here.
Throughout the process of creating and edit data sets you’re going to spend a lot of time in Interactive System Productivity Facility (hereafter ISPF). ISPF sits on top of TSO (ie what you logged into) and gives you a user interface that is supposed to be more friendly (ehh...)
Yak 01: You’re going to use fn A LOT (Working in ISPF)
Working with ISPF is … interesting. I feel like every time a Vim -vs- Emacs argument breaks out over the internet all participants should be required to use ISPF for a period of time just so they appreciate what we have in modern text based editors.
ISPF’s design sort of feels like it really wants to be a GUI, but it’s not a GUI and you have to use the keyboard to navigate it. That gives it certain … quirks.
The first screen you will be presented with upon logging in is the Option Menu which looks something like this:
By default the command prompt is at the bottom of the screen, here labelled with Option ===> This part should be self-explanatory, you type the letter/number that corresponds to the option you want and hit return. But it gets a little wonky when you realize that if you press the up key the cursor will wonder off the command line and let you move around the screen itself. If you type your option while not on the command line your machine will beep in disapproval. If you navigate your cursor over to option you want in order to select it, your cursor will jump back to the command line.
ISPF doesn’t seem to care where on the command line you type commands, just that you do certain commands on the command line. So it you want to slide your cursor all the way to the right before typing in 1 to view a file, go ahead.
The other major thing to know about ISPF’s interface is that you are going to work your fn key a lot. So know where it is and get comfortable with that finger position. The F-keys you’ll be using the most in this exercise are as follows:
F3: Exits out of whatever screen you’re on and moves back to the previous screen. If you are editing a data set this will also automatically save your work before exitingF8: Scroll downF7: Scroll up
Some options in the menu have submenus. If you know that you want to open the Foreground (option 4) processing menu and select the COBOL compiler (option 2 on submenu) you can just type in 4.2 on the command line and jump straight there.
Yak 02: Allocating a Data Set for COBOL
If I had to pick the thing that caused the most angst it was figuring out what settings my data set should have in order for my COBOL code to compile correctly.
Actually … no, that wasn’t the worst part. The worst part was figuring out the reason why my program would not compile was because my data set settings were WRONG. You may notice in some of my screenshots that I’m using group names like COBLOG3 and COBLOG4. That’s because I fucked this up and I fucked this up a lot of different ways.
To allocate or create a data set you go to the Utilities panel (option 3) and select the Data Set option from the submenu (option 2). The first screen will be pretty straight forward, you navigate your cursor over to the fields for Project, Group, and Type and type in ibmuser, your folder name and COBOL respectively. Then you navigate back down to the command prompt (you can use your mouse to do this, FYI) and type in the option you want (in this case A for allocate)
Then you are presented with a screen like this:
….And will be distressed to find out that you can’t just go ‘Pfftt… whatever’ and not set these parameters. Also, fun fact, most of the parameters cannot be changed once the data set is allocated!
If, like me, you went to the official IBM documentation looking for some guidance on what these values should be, you probably ended up on this page and gave your first data set settings that are very very bad for COBOL. Here are the places where you’ve now screwed up and what error messages you will likely get when trying to compile:
RECORD LENGTHneeds to be 80. Remember COBOL originally ran on punch cards that were 80 columns in width, so COBOL programs are fixed width and 80 columns. If you setRECORD LENGTHto something larger than 80 the compiler will tell you “input LRECL too large for language type”. LRECL stands for Logical Record Length and it took me forever to figure out what this had to do with anything. Weirdly, the solution came when I got stuck in a nasty error loop in ISPF’s command line over a rouge hyphen. Quite at random z/OS spat out an message telling me “BTW LRECL for JCL programs needs to be 80” and I made the connection.BLOCK SIZEshould be 4000. Well, okay, this one was my own fault. I knew by this point that Block Size needed to be a multiple of LRECL and I just wasn’t pay attention and kept it set at 5100. Your Block Size can be different if you want, as long as it divides evenly by 80.RECORD FORMATthis should be set to FB (fixed block), not VB (variable blocks) for reasons that should be obvious giving everything described above. Getting this wrong created some really interesting chaos on compile. At first it looked like z/OS COBOL compiler was just a little more anal about indentation than GnuCOBOL is, but after a couple rounds of fiddling with the indentations I realized that the compiler was ignoring line breaks, producing the following errors:
Once you’ve got those settings right the only other thing to remember is to specify as your Data set name type to PDS for Partitioned Data Set.
And away we go!
Yak 03: Line Commands -vs- Command Commands
Data set properly allocated it’s time to start writing some COBOL.
On the main option menu select 2 to edit your data set and fill in the Project, Group, and Type values of your new dataset. Then enter in a file name on the line for Member and a blank file will open.
If it says “X Protected” when you try to edit the file, go back to the last screen and select the box that reads “Edit on Workstation”
If you have messages on the top of the file you can clear them by typing res or reset into the Command prompt on the bottom, but that will also clear all your empty lines. Not a big deal right? You’ll just move the cursor to the end of the “Top of Data” heading and hit Return.
Except that’s not how you create new lines in ISPF. The editor has two columns, the first on the left with the red text is a margin. As you type out your program it will auto-populate with line numbers. But it’s also where you type in line commands. Remember when I said that ISPF was quirky you had to make sure you were typing the right command in the right place? This is what I was referring to. When you want to create a new line break in ISPF you do not enter a command in the main command prompt, you go to the line directly before where you want the new line and type i anywhere in that red margin. Then you hit return and ISPF will create a new line and restore whatever the value of the margin was supposed to be.
Well let’s write up our test COBOL program with some errors and fix them with ISPF:
If I tried to compile this program it would error because the indentation is wrong. Certain things go in certain columns in COBOL. Things like division headers (IDENTIFICATION and PROCEDURE) need to start at col 8, here I started at col 4. So I need to shift all my lines over by 4 spaces. Luckily there’s a line command for that. I can type in )4 to shift an individual line or )) on the first line I want shifted followed by))4 on the last line I want shifted to move over the whole block.
We also can’t call our program ‘Test’ (presumably that word is reserved) so we have to change it to something else. Unfortunately I have not figured out a good way to insert characters in the middle of a line. I’m having trouble believing that there is no way to do this other than retyping the whole line over, but I haven’t been able to figure out what the command for it is. For the time being I’ve been using the change on primary command prompt which allows you to do a find and replace. Entering change TEST HELLOW will do the trick.
There are more line commands and primary commands in ISPF than I have time to detail. See this write up for a nice list.
Yak 04: Compile, Link, Load
It’s probably not fair to refer to z/OS as legacy, but it definitely retains some legacy paradigms which take some getting used to. When it comes to compiling and running a program you have to tell the mainframe to do each step, it assumes nothing, whereas modern development tools tend to default towards automating the steps you’re most likely going to want to do together and adding flags to break things out if that’s desired.
The traditional way of doing this on z/OS is using IBM’s Job Control Language (JCL) which is something like the bash of mainframes. In fact every shred of documentation will focus on how you do things in JCL, ignoring the interface that ISPF provides for such tasks.
We’re not going to use JCL here. Or rather, we are going to use JCL but we’re going to let ISPF generate it for us. We’re going to do it this way because JCL is pretty obtuse looking at first glance and we want one less yak to shave in this process.
There are three steps we need to do to run what we have now in our data set. First we need to compile it. z/OS’s COBOL compiler will take our code and turn it into what’s called an Object Deck (a callback to the old days when your compiled output was punched out on a deck of punch cards). To do that we’re going to use the z/OS Foreground compiler for COBOL (or option 4.2 from the main menu)
The panel that comes up will ask for your Project, Group, Type, and Member. You can leave everything else blank or at the default. When the compiler is finished you’ll see three red stars as a prompt then ISPF will automatically load the job output. Hit F8 a few times to scroll down and check for any errors. If everything went well you should have a return code of 0.
Now go back to the main menu, select view and change the type from COBOL to OBJ. Leave the member name blank and keep your Project and Group the same as your dataset. You will see the name of your program listed among the object decks.
But object decks themselves are not executable programs. This goes back to how these machines originally worked. Your object deck was a physical deck of punch cards. It wasn’t a program that could be run into you loaded it back into the machine. So the next thing we need to do is generate a load module.
If you go back to the main menu and select Foreground (option 4) again you should see in addition to various compilers you can also select something called a Binder/Link editor. This is what will generate our load module.
First set of parameters should be familiar by now, we need to type in the location of our object deck. Second section is whether we want to use the Binder or the Linkage Editor. The difference between them is not super important right now. It defaults to Binder but for a Hello World it’s probably better to use Linkage.
When you’re creating a load module the Linkage Editor needs to make sure to include the program’s dependencies. COBOL like all other high-level programming languages requires certain libraries and the compiler does not automatically package them up with the object deck. Specifically, what you need is in a data set called SCEELKED, but finding it might be a challenge.
It is most likely at CEE.SCEELKED and since ibmuser is effectively a superuser account, there’s a really easy way to verify this. We navigate back to the main menu, select Utilities (3) and Dslist (4) then on the line that says Dsname Level we type in CEE and leave everything else blank.
This will pull up a full list of everything under the CEE project. If SCEELKED is not there, check SYS1 or SYS2
If you fail to include the SCEELKED library when you try to create a load module you will get the following errors:
SYMBOL IGZCBSO UNRESOLVED
SYMBOL CEESTART UNRESOLVED
SYMBOL CEEBETBL UNRESOLVED
SYMBOL CEESG005 UNRESOLVEDIf the load module is generated correctly you will get an Authorization Code of 0 and see the module in Project.Group.LOAD(Member)
There is a much easier way to interact with the Linkage Editor and that is directly through TSO. Select Command on the main menu (Option 6) and at the prompt enter LINK project.group.OBJ(Member) lib('cee.sceelked'). It does the same thing without ISPF’s interface.
So now (finally!) we’re ready to execute our program. We go the TSO Command line (option 6) and enter the following command:
CALL 'Project.Group.LOAD(Member)'
Yak 05: Working with Spools (Bonus Yak!)
There are some inefficiencies to our approach here. JCL scripts not only allow us to do all three steps in one job, but the output of one can be directed into the input of the other thereby creating temporary data sets where this manual process created more permanent ones. By using the TSO command CALL our output is just dumped directly to stdout, but a normal COBOL program that is compiled and run with JCL would likely send its output to the spool. Even though we don’t cover JCL in this post, it’s worth taking you through the spool interface anyway. It will come in handy later on.
Option S on the main menu, the Spool Display and Search Facility is where you’ll find your output most of the time. You can also look at system logs, see what jobs are in the queue and their status, as well as how the mainframe’s resources are being used.
The two queues to pay attention to in general are the Output Queue and the Held Output Queue. The held queue is where your job output will probably end up when there’s an error. The output queue is where successfully completed jobs tend to go.
When browsing output queues, there are some useful line commands to remember. S will select the file on that line. ? will allow you to jump to specific component parts of the output (which can be useful when the output is long and you know exactly which section you want to look at). Often a job’s output is more than just the result. Statistics on how the job ran, what it did and where it did it are also part of the output found in the spool.
Going Forward
Most of this stuff was pieced together through trial and error. I will update this post as my understanding evolves. I’m sure the simple act of writing this all down and putting it on the internet will bring out all sorts of people who will contribute more detail. Corrections and clarifications are welcome in the comments!
