File Master™ Plus enables Mainframe Modernization in Place
File Master Plus’ Populate REST API allows us to keep on the Mainframe modernization path with COBOL applications.
Background
The Company’s core applications have been stable for a long time. The applications were written using CICS, COBOL and DB2 and have not required new applications to support the processes. IT decided to outsource mainframe development. This outside organization provides bug fixes, new functions and enhancements to the existing applications. They consider it maintenance.
The DRAGON application was developed in the 1990s. It is a set of COBOL programs and copybooks. It processes hundreds of sequential files sent from many external sources. Those files are uploaded to the mainframe after being properly encoded. The files match the COBOL copybook layouts.
Scenario
We were recently notified the data format will change. Data providers will now be sending the data in JSON encoded files instead of the previous formats.
JSON stands for JavaScript Object Notation. It is a standard file format used to interchange data. Data objects are stored and transmitted using key-value pairs and array data types and they are human readable.
Well, we have an issue …
Although it is not a big change … it is for us:
· The application hasn’t changed since 2001, when it was adapted to work with EURO currency.
· The current outsourced development team has never worked on this application. The original team has long been retired.
· There is no documentation.
· COBOL version compatibilities.
· The application has never been under version control with a tool like Endevor. This means Impact Analysis can’t be performed.
· There is a company old legend saying that the source code is missing!!
I know it may look a bit exaggerated, but I’ve heard all these things one time or another.
The Plan
We have engaged a team of people to determine the best approach to moving forward.
The first idea is request the outsourcer to modernize the application to adopt JSON. With IBM Enterprise COBOL V6 compiler, you can generate and parse JSON documents directly from COBOL. Starting from V6.1, it supports JSON GENERATE, which converts COBOL data items to JSON text. From V6.2, it also supports JSON PARSE, which converts JSON text to COBOL data items.
But the project scope isn’t fully defined. We aren’t sure how many programs will need to be modified. We also don’t know how long it will take to modify and test the application. We can’t really define the cost and we certainly haven’t budgeted for this, either.
A set of applications were updated to support XML data years ago. It was a huge, costly effort.
z/OS Connect and z/OS Client Web Enablement were evaluated. They can convert JSON text to binary data. But they weren’t the right approach for us.
Modernization In Place
With new people in our IT department, we started embracing DevOps practices in our Mainframe. This younger generation brought a set of skills and productivity tools that are now part of our architecture.
One of the young system programmers aware of this issue with the DRAGON application suggested using the FMP Populate RESTful API.
FMP stands for File Master Plus It is a tool that allows file editing and data creation for mainframe data set files, as well as IMS/DB and DB2 for z/OS databases among other functionalities.
API stands for Application Program Interface. It is a way for two or more computer programs to communicate with each other.
REST stands for Representational State Transfer. It is an architectural standard for an interface between different components.
This FMP feature populates a data stream in an array in JSON format to a member, sequential or VSAM file mapped as a given layout (COBOL copybook in this case).
WOW … it seems that this could be a nice solution for us. Setting up a pipeline to process the JSON files as records the DRAGON understands. Let’s do some testing!
This is our sample copybook. It resides in our Mainframe:
VIEW DRAGON.LIB.COPY(MARBLES) - 01.00 Columns 00001 00072
Command ===> Scroll ===> CSR
****** ***************************** Top of Data ***************************
000100 01 MARBLES.
000200 05 COLOR PIC X(10).
000300 05 FILLER PIC X.
000400 05 INVENTORY PIC 9(3).
000500 05 FILLER PIC X.
000600 05 COST PIC 9(3).
****** **************************** Bottom of Data *************************This is our sample data in JSON format:
[
{
"COLOR": "RUBY ",
"INVENTORY": "000",
"COST": "000"
},
{
"COLOR": "TURQUOISE ",
"INVENTORY": "025",
"COST": "005"
}
]To populate the JSON file into a PDS member named MARBREC in DRAGON.LIB.DATA, we take a look at the FMP Populate REST API in the documentation and here is the structure we could use:
PUT /fileMasterPlus/api/v1/mvs/dataSets/DRAGON.LIB.DATA
{
"request": "populate",
"member": "MARBREC",
"layout": ["RODDI01.LIB.COPY","MARBLES"],
"data":[
{
"COLOR": "RUBY ",
"INVENTORY": "000",
"COST": "000"
},
{
"COLOR": "TURQUOISE ",
"INVENTORY": "025",
"COST": "005"
}
]
}Postman is a great help when testing APIs.
Postman is one of the most popular software testing tools which is used for API testing. With the help of this tool, developers can easily create, test, share, and document APIs.
Here is the desired result:
VIEW DRAGON.LIB.DATA(MARBREC) - 01.00 Columns 00001 00072
Command ===> Scroll ===> CSR
****** ***************************** Top of Data ****************************
000001 RUBY 000 000
000002 TURQUOISE 025 005
****** **************************** Bottom of Data **************************How are we going to trigger this call? There is good news. There’s no wrong way to do it. Any tool in our production environment can work: any workload scheduler, Jenkins automation pipelines, service portals, event managers …
We can easily execute a ‘CURL’ command or use any scripting language to make the API call. Here is a Python example:
import requests
import json
payload = {
"request": "populate",
"member": "MARBREC",
"layout": [
"DRAGON.LIB.COPY",
"MARBLES"
],
"data": [
{
"COLOR": "RUBY ",
"INVENTORY": "000",
"COST": "000"
},
{
"COLOR": "TURQUOISE ",
"INVENTORY": "025",
"COST": "005"
}
]
}
headers = {
'accept': "application/json",
'content-type': "application/json"
}
r = requests.put('http://IPO1:5194/fileMasterPlus/api/v1/mvs/dataSets/DRAGON.LIB.DATA', data=json.dumps(payload), headers=headers)
print(r.text)It is possible to use FMP with Ansible as well.
Ansible is an open-source IT engine that automates application deployment, intra service orchestration, cloud provisioning, and many other IT tools.
- name: Populating a data set
broadcom.file_master_plus.populate_data_set:
host: ipo1
port: 5194
name: dragon.lib.data
member: marbrec
layout_member: marbles
layout_data_set: dragon.lib.copy
data: ../instream/data1.txtBut if I had to choose I would select, in my humble opinion, I believe Zowe CLI is the game changer for Mainframe Modernization. The FMP plug-in for Zowe CLI allows application developers script DevOps processes from a command line interface. Zowe will connect to the REST services available in our Mainframe via the corporate authentication method.
Sample Zowe CLI FMP command:
zowe fmp populate ds DRAGON.LIB.DATA -m MARBREC --lds DRAGON.LIB.COPY --lm MARBLES --data ./data/marbles.jsonI could share more examples, but I’ll leave it at this point. The whole purpose of this post is to address how important is to have a knowledge of the tools we have available, be updated and get the max from them.
If you enjoyed this blog, check out the Modern Mainframe and Zowe publications. I also recommend Thomas McQuitty’s blogs.
