Automating CA Endevor Packages With Zowe CLI
CA Endevor is a DevOps Tool
CA Endevor is a market leading mainframe DevOps tool and has been for many years. The very name Endevor actually stands for Environment for Development and Operations. A key DevOps practice is being able to easily and continuously build changes and shepherd them through a defined life cycle or pipeline. Given that, it should be no surprise that CA Endevor provides this capability.
The compile process for mainframe applications does not generally produce bundles of executable code for deployment as a complete application. For example, with java-based applications, artifacts such as jars, wars and ears (i.e. complete application bundles) are created during the build and deployed to different environments at various stages in the pipeline. By contrast, the mainframe compile and link process produces load modules with loose dependencies. Tools like CA Endevor are used to identify the dependencies and bundle them, along with the executables, and move them through the pipeline and its associated environments. The vehicle for this in Endevor is called a package.
Packages Defined
Traditionally, mainframe developers have had the responsibility of defining packages based on the changes they make. In many cases, the packages they use to promote from development and deploy to QA are not the same as the ones used for higher environments. Packages for higher environments need to account for dependencies as well as defect fixes moved in after the initial change. In the world of modern mainframe though, organizations are looking to automate this activity in the interest of improving velocity and relieving developers from this burden.
To understand how we can begin to automate packages for CA Endevor, let’s first look at what a package is. A package, simply put, is a series of instructions to do something in Endevor. The instructions are specified in a language called SCL. It can include things like adding or building elements but more typically though, they are a series of move instructions to advance changes through a defined life cycle. Developers generally decide what they want to move, then interactively build or create the package commands using a menu-driven interface that allows them to choose what to move, and from where. When they are ready to act on the instructions, they perform an operation called cast which validates the SCL and ensures required dependencies are included in the package and any potential version clashes have been resolved. After the cast operation, the package and the affected elements are frozen until the package instructions have completed. Once completed, the package is marked with the Executed status. The package may also require approvals before operations can begin.
In general, packages operate entirely within the CA Endevor managed datasets. Very frequently though, the binaries need to be delivered to additional, highly secured non-Endevor managed datasets after being moved up the life cycle in the Endevor environment. This can be achieved by shipping the package to an external destination known to and authorized for access by Endevor.
Automating Packages in a Pipeline
In terms of automation then, we need to be able to support the following operations:
- Defining the initial package
- Updating package SCL — similar to what a developer might do interactively, where the SCL is newly created or added to existing SCL defined for the package
- Casting the package
- Approving the package — potentially by the pipeline itself after approval is obtained in another tool like a change control system
- Executing the package
- Shipping the package
In a pipeline context you may very well want to repeat the above steps at each phase of the life cycle, woven in with non-Endevor tasks like change control operations or automated / manual testing. One of the nice gains you get from automating your package process is the fact you can put all these steps together as part of an end-to-end workflow, regardless of what platform the steps are performed on.
Zowe CLI facilitates all of this by providing command line operations that can perform each of the required tasks. Simple shell scripting combined with an overall orchestration tool like Jenkins Pipeline creates the end-to-end workflow. If you are looking for a solution that requires less scripting, you can find purpose-built integrations similar to what is being described here for Broadcom’s own Automic Continuous Delivery Director, which is powered by Zowe CLI.
Defining Packages with Zowe CLI
Let’s look at each of the operations in detail, starting with defining a package and generating SCL for it. To define a package in Zowe CLI, you would use the following command:
zowe endevor create package Package_Name -d Package_Description -i Endevor_Instance --novalidate-scl
For example:
zowe endevor create package MYPKG -d "Its a package" -i ENDEVOR --novalidate-scl
Note: the --novalidate-scl — this is used to create an empty package initially so we can query for elements to add to the package and generate the SCL we will use.
You may be wondering how that command is connecting to the mainframe — for instance, what LPAR is this being run on and what user ID is used? Those are all things stored in a secure Zowe CLI profile. The profile tells Zowe CLI which Endevor web service host and port to connect to, along with some other parameters like username and password. The web service host, port and instance (the -i parameter above) effectively determine the LPAR you are running on. You can have multiple profiles and specify which one to use via the settings for default profile or you can pass it as a command line parameter. You can also specify the profile settings explicitly on the command line if preferred. To review the capability around profiles, use the Zowe CLI profiles command group:
zowe profiles --help
In order to create the SCL to use with the package, you first need to get some details about the elements you want to add. Developers will typically know the elements they changed and be able to find them quickly to build SCL in ISPF, but in an automated setting, the human element is not present. This means we need some sort of way of finding out what we changed. A natural candidate for that is the Endevor CCID (Change Control IDentifier) of the change. A CCID is a tag that is meant to associate an element update with the ID of a related change control artifact in another system. We can query for changes in a given inventory location in Endevor and use the results to build the SCL. If our pipeline allows parameterization, we might parameterize the CCID or even other inventory location values or maybe we will hard code the inventory locations for our application in our pipeline.
To search for elements by CCID, you would use the zowe endevor list ele command. In addition to the inventory location parameters (e.g. --environment --system --subsystem --stage-number --type), you can add filters for CCID as follows:
So for instance, if you wanted to find all the cobol elements that were changed with CCID = US227 for the last action, in system = FINANCE, with subsystem = ACCT* in environment = SMPLTEST and stage number = 1, you could use the following zowe command:
zowe endevor list elements --env SMPLTEST --sn 1 --sys FINANCE --sub ACCT* --type COBOL --where-ccid-lastact US227 --sm -i ENDEVOR
The output from the command will look something like this:
FINARP01 COBOL SMPLTEST 1 FINANCE ACCTREC
FINARP02 COBOL SMPLTEST 1 FINANCE ACCTREC
FINARP03 COBOL SMPLTEST 1 FINANCE ACCTREC
FINARS03 COBOL SMPLTEST 1 FINANCE ACCTREC
FINARS04 COBOL SMPLTEST 1 FINANCE ACCTREC
….Of course, this is not SCL — so now we need to parse the output and turn it into proper SCL syntax before we update the package. HINT: The --sm in the example will make the output easier to parse to turn it into SCL.
A very common use case is to create packages to perform the “Move” action in Endevor. The syntax for moving an element is as follows:
MOVE ELEMENT ‘Element_Name‘
FROM ENV ‘Environment_Name’ SYS ‘System_Name’ SUBS ‘Subsystem_Name’
TYP ‘Type_Name’ STAGE NUMBER Stage_Number
.NOTE: there are other options such as CCID, Bypass Element Delete, With History or Retain Signout that you may wish to include. As well, the period shown above must be included at the end of each SCL statement. Details can be found here.
HINT: The syntax for building elements is very similar to the above, essentially substituting GENERATE ELEMENT for MOVE ELEMENT.
To turn the output from the list command to SCL, you would use a simple script, leveraging regex to parse the output and write a temporary SCL file with the correct SCL syntax. Here’s a sample of what that might look like in bash, starting from the define package command:
#!/bin/bash
#
# Define package script for a static app inventory location
# and using a given package name and CC ID to query for elements
#
# define_pkg.sh package_name cc_id
## Define an empty package
zowe endevor create package $1 -d "$1" --novalidate-scl# Query for cobol elements by CC ID in FINANCE/ACCTREC in SMPLTEST/1
# Store results in a temp file
TMPFILE=$(mktemp)
zowe endevor list elements --env SMPLTEST --sn 1 --sys FINANCE --sub ACCTREC --type COBOL --where-ccid-lastact $2 --sm > $TMPFILE# Read the file and translate to an SCL syntax temp file
SCLFILE=$(mktemp)
regex='([[:alnum:]]*)[[:blank:]]*([[:alnum:]]*)[[:blank:]]*([[:alnum:]]*)[[:blank:]]*([[:digit:]])[[:blank:]]*([[:alnum:]]*)[[:blank:]]*([[:alnum:]]*)'
while IFS= read -r line; do
[[ "$line" =~ $regex ]]
echo "MOVE ELEMENT ${BASH_REMATCH[1]}" >> $SCLFILE
echo " FROM ENV '${BASH_REMATCH[3]}' SYS '${BASH_REMATCH[5]}' SUBS '${BASH_REMATCH[6]}'" >> $SCLFILE
echo " TYPE ${BASH_REMATCH[2]} STAGE NUMBER ${BASH_REMATCH[4]}" >> $SCLFILE
echo "." >> "$SCLFILE"
done < "$TMPFILE"zowe endevor update package $1 --from-file "$SCLFILE"rm "$TMPFILE"
rm "$SCLFILE"
HINT: The above script is a simple shell script, however, you may use any scripting language you want with Zowe CLI, such as python or javascript. Most commands also allow you to pass the --rfj option that formats the output in JSON making for easy parsing in modern languages.
Performing Package Operations with Zowe CLI
Most of the other operations are simpler, with simple Zowe CLI commands mapping to each operation. For example to cast a package, you use the following command:
zowe endevor cast package Package_Name -i Endevor_Instance
To approve a package, you would use this command:
zowe endevor approve package Package_Name -n “Approval_Notes” -i Endevor_Instance
If you were rejecting the package, you change the approve action to “deny”:
zowe endevor deny package Package_Name -n “Approval_Notes” -i Endevor_Instance
To execute the actions in a package, you would use the following command:
zowe endevor execute package Package_Name -i Endevor_Instance
Note: for long running packages you can also use the zowe endevor submit package command. As well, for all the commands above, there are different options for each command I would encourage you to explore, depending on the needs for your pipeline.
Shipping Packages with Zowe CLI
The last operation to cover is Package Ship. The Package Ship facility is available in Zowe CLI via the zowe endevor submit scl command, which takes input from an SCL file on the machine the script is running on. This means we may need a bit more scripting to perform a ship operation.
Package Ship SCL has the following format to ship the original package binaries:
SHIP PACKAGE Package_Name TO DESTINATION Destination_Name OPTION OUTPUT .
Or, to ship the backout binaries, you would use:
SHIP PACKAGE Package_Name TO DESTINATION Destination_Name OPTION BACKOUT .
Note: multiple ship statements in a single submission is not allowed with the Ship API so you need to submit each one at a time. Let’s assume an application has Package Ship locations in Chicago, Dallas and New York. Here is a snippet of script that would ship a given package name (passed as a parameter to the script) to those locations via Zowe CLI:
#!/bin/bash
#
# Define package ship script for a static ship destinations
# and using a given package name
#
# ship_pkg.sh package_name
#
# Create a temporary SCL file
SCLFILE=$(mktemp)# Run Ship SCL statements
echo "SHIP PACKAGE $1 TO DESTINATION CHICAGO OPTION OUTPUT . " > $SCLFILE
zowe endevor submit scl --scl-file "$SCLFILE" --scl-type ship -i ENDEVORecho "SHIP PACKAGE $1 TO DESTINATION DALLAS OPTION OUTPUT . " > $SCLFILE
zowe endevor submit scl --scl-file "$SCLFILE" --scl-type ship -i ENDEVORecho "SHIP PACKAGE $1 TO DESTINATION NEWYORK OPTION OUTPUT . " > $SCLFILE
zowe endevor submit scl --scl-file "$SCLFILE" --scl-type ship -i ENDEVOR# Clean up
rm "$SCLFILE"
HINT: The zowe endevor submit scl command will also allow other types of SCL to be run, including admin operations, making it a powerful way to incorporate CA Endevor operations that we don’t yet have dedicated commands for.
Summary
Hopefully this has given you a good idea of how you can start building package automation into your CI/CD pipelines. By doing so, you can:
- Create cross-platform pipelines while at the same time leveraging existing automation you already use for deployments
- Weave in other tasks so the automation you have can become an end-to-end workflow, even when the added tasks you want to automate run on other platforms such as automated GUI testing
- Use the visualization and auditing that comes with modern CI/CD tools
- Use webhook consumption to respond to events and drive downstream processing
Learn more about Zowe at this site; more Zowe blogs here.
For more information on the CA Endevor Plug-in for Zowe CLI visit: CA Endevor Zowe CLI Plug-in Documentation.
As well, if you have any comments, questions or suggestions on the above, I’d love to hear from you — send me an email at Vaughn.Marshall@broadcom.com
