Continuous Integration for a Mainframe App
{Ecosystem} Have you lost your marbles? 😱
Actually, no, thanks to the continuous integration pipeline built for our mainframe application! 😎
In this blog, I will discuss how to construct a CI pipeline for a sample mainframe application using modern open-source tooling including the Zowe CLI. First, I will introduce the application that we will be building the CI pipeline for. Then, I will focus on automating the build, deploying to a test environment, and testing of the application. Finally, we will focus on orchestrating this automation via a Jenkins pipeline.
The sample application created for this exercise is called Marbles. The Marbles application tracks the inventory of marbles of various colors. For example, in the image below we have 3 RED marbles in our inventory.
The Marbles application uses the following systems to function:
- IBM Db2 Database
Stores the data to track the color and quantity of marbles. - IBM CICS Transaction Server for z/OS
Handles input from the user to change the quantity or color of marbles in the database. - IBM WebSphere Application Server (WAS) Liberty
Hosts the code that is contained in a deployed Web application Archive (WAR) file so that users can access the application at a given URL. - IBM WebSphere MQ
Handles the communication between WAS Liberty and the CICS region that handles transactions.
I want to focus on the build (compile and link), deployment, and testing of the CICS COBOL code. The code for this application is stored in a CA Endevor Software Change Manager repository. CA Endevor processors exist for building (generating) the application. Members are created in a DBRMLIB and LOADLIB. After a code change is built, these members need to be deployed to a test environment, JCL needs to be run to perform the Db2 BIND & GRANT functions, and a CICS CEMT transaction needs to be issued to load a new copy of the CICS program. Finally, the CICS transaction needs to be tested.
My goal is to automate everything beyond the code change. I will use modern open source tooling to accomplish this. If you are interested in following along, all the automation code is available here! 😍
For the build and deployment to a CI environment, I elected to use the Zowe Command Line Interface and various Zowe CLI conformant plug-ins to drive interaction with z/OS. For help getting started with the CLI, check out “Getting Started with Zowe CLI”. I also decided to use Gulp to abstract simple commands into more complex tasks. Once you become proficient with the CLI and create a few functions, it becomes very easy to build out custom gulp tasks. As an example, let’s look at the task to generate the COBOL element and the task to perform the BIND and GRANT.
The submitJobAndDownloadOutput function accepts a data-set to submit, a local directory to download spool output to, a maximum allowable return code, and a function to call after completion (in this case we just pass the callback that signals the completion of the gulp task).
The simpleCommand function accepts a command to submit, a local directory to download command output to, and a function to call after completion. If you wanted to check for some expected output from the command, the function also accepts an array of expected strings to be in the output as an optional argument.
As you can see, it is fairly straightforward to automate additional tasks. Once the tasks are written, it enables us to build and deploy our application very efficiently. Since we log all the output from our gulp tasks, it is also easy to debug if needed.
For the testing of the application, I chose the Zowe CLI to drive interaction with z/OS and Mocha, a popular JavaScript test framework, to build my tests.
The sample test below verifies that the CICS transaction creates a single marble of a particular color with a cost of one. The getMarbleQuantity function uses the Db2 plug-in for Zowe CLI to query the table to retrieve the inventory and cost of the desired marble color.
Once the test automation is written, it is simple to exercise and we can view graphical test results without writing our own UI.
Once we created build, deploy, and test automation that could be run by individual developers, we wanted to standardize this as part of a continuous integration pipeline to ensure the tests passed after each code change. We chose Jenkins as our CI orchestration tool.
Jenkins pipelines can be defined in source control via a Jenkinsfile. Including your pipeline in source control allows for code review and iterative improvement as well as an audit trail of how the pipeline has changed. The Jenkinsfile for our project consists of a local setup, build, deploy, and test stage. The pipeline runs in a docker container labeled zowe-agent which already has the CLI installed. The local setup stage installs some additional development tools/packages and creates the profiles used to interact with various services. Notice that dummy values are supplied for the host, user, and password options. This is because we would not want to source this sensitive information. Instead, this information is stored as Jenkins secrets and only accessed when needed. The build, deploy, and test stages make use of the automation we have already written as shown below.
In addition, the pipeline’s activities are archived via the following code:
In the end, we have the following Jenkins pipeline that provides for transparency and drives incremental improvement for our continuous integration efforts.
Thanks for hanging in there! If you would like to make use of this sample, check out the repo! You can also learn more about Zowe at this site or read other Zowe blogs here.
