Running Zowe CLI on Jenkins with Docker

{Core} In previous story Better Mainframe Automation using Zowe CLI and Python, we have seen how one can embed The Open Mainframe Project’s Zowe CLI commands into Python and write more complex automation.

Good automated scripts will not be stored just on your computer and executed by you. You would like everyone to be able to run them when they are needed, have them scheduled, or kicked off when something else happens (code has changed, job ended…). This is the point when a tool like Jenkins can help you.

What is Jenkins? It a popular open source automation server that helps you with building, deploying, and automating any project. For information, see its homepage at https://www.jenkins.io/. Jenkins provides multiple ways how can define the jobs and pipelines that are executed by it.

In this text, we focus on Jenkins pipelines and how to run Zowe CLI commands from such pipelines. If you have not heard about Jenkins pipelines, I recommend you to have a look at the following video: Complete Jenkins Pipeline Tutorial | Jenkinsfile explained — YouTube.

Jenkins is not the only tool that can help you to automate your pipelines. There are other tools like GitLab, GitHub Actions, CircleCI, or TeamCity to name a few. A lot of them can use containers as well and a similar approach as described in this story can be used with them too.

The following examples are for Zowe V2. You can learn more about the major changes for the CLI in V2 in Zowe CLI — Getting Started, Made Easy! | by Michael Bauer | Zowe | Medium.

Photo by Possessed Photography on Unsplash

How to Setup Zowe CLI on Jenkins

There are three types how to approach the installation of Zowe CLI on Jenkins:

1. Global Zowe CLI installation and configuration on Jenkins server

This is an easy way how to start. You have one server with Jenkins and you just install Zowe CLI and its plugins in the same way as on your computer. The Zowe CLI and its configuration is global to the system so any Jenkins pipeline running on that server can use it.

Problems start when you need to scale it and have, for example, additional agents or you need to use different version of Zowe and plugins or special configuration for each pipeline.

This is a good option to try few things with Zowe but not a good basis for reliable usage of Zowe CLI in the long-term.

2. The Zowe CLI and plugins are installed and configured in isolated workspace of the pipeline

We are moving close to “configuration as code” approach. Configuration as code is the practice of managing configuration files in a code repository. For example, your automation scripts are stored in a Git repository. In approach #1, those script would rely on a configuration of a particular Jenkins server and might not work well or at all at different Jenkins server.

With approach #2, the Zowe CLI is not installed on the Jenkins server globally but it is temporarily installed and configured as a part of the workspace for each pipeline. The script and its configuration to do that are stored in the same Git repository. Once the pipeline is completed, its workspace is discarded including its installation of Zowe CLI.

TheZOWE_CLI_HOME environment variable is used to have a local Zowe CLI environment in the same directory as your workspace. However, it does not provide full isolation of your pipelines.

This extra setup requires more time but it gives you the flexibility and you can run the pipeline on Jenkins servers or agents that do not have Zowe CLI installed and have different configuration for each project. The only requirement is to have Node.js installed on the system. In reality, there will more requirements that do know about until you try to migrate to a different system.

3. The pipeline uses a container (Docker image with Zowe CLI)

Container technology like Docker provides a way how you can package any application or a tool together with all of its dependencies including runtimes like Node.js, the operating system, and its configuration into a single reusable image. Such image is ready to be started and provides immediately a running instance of your application.

There are many benefits that apply to our use case. Each pipeline execution will get a consistent and fully isolated environment. It will always start from the image that you specified and what happens in the container stays in the container. The container can run on any server that has Docker or similar tool installed and in starts in seconds.

We will cover steps how to create Docker image with Zowe CLI and how to use it Jenkins.

If you have never used Docker before, it might useful to learn something about it. For example, from Docker Tutorial for Beginners — YouTube. We will try to explain all the steps.

Dockerfile with Zowe CLI

First, we need to create image that we can use to run Zowe CLI commands. Images are defined in a special format called Dockerfile. It basically says how the image is build.

We will write such Dockerfile from scratch.

The first line specifies a base image. We will pick an official Docker image of Node.js. Release 16 is an LTS (long-term support) release. Zowe CLI supports all supported LTS releases of Node.js:

FROM node:16

This image is based on Debian distribution of Linux version 10 (“buster”).

We will put all the Zowe-related files into /zowe directory:

# Directory where Zowe CLI settings and plug-ins will be stored:
RUN mkdir /zowe
WORKDIR /zowe
ENV ZOWE_CLI_HOME=/zowe

Zowe CLI itself has additional dependencies that need to be installed to the Linux system:

# Install requirements of Zowe CLI Secure Credential Store:
RUN DEBIAN_FRONTEND=noninteractive apt update
RUN DEBIAN_FRONTEND=noninteractive apt install -y libsecret-1-dev gnome-keyring dbus-x11

Then we can install Zowe CLI:

# Install Zowe CLI:
RUN npm install -g @zowe/cli@next

Note that we are using the @next version of Zowe that is the V2 version. Then you can add CLI plug-ins that you need:

# Install Zowe CLI plug-ins:
RUN zowe plugins install @broadcom/endevor-for-zowe-cli@next

For the complete list of Zowe CLI plug-ins that you can use, see Zowe Conformance Program — Open Mainframe Project.

Zowe CLI V2 has a new good feature — daemon mode. It significantly improves performance of the Zowe CLI. More more information see:

Zowe CLI — A Faster Experience (Made Better)

To enable it, we will add following lines:

# Enable daemon mode:
RUN zowe daemon enable
ENV PATH=/zowe/bin:${PATH}

The container needs to have input from the outside. Typically, you will submit commands into it and run existing scripts that are stored on your computer. For that we will mount this directory to /workspace directory in the container. This will prepare it and set is as the current working directory in the container:

# Working directory for the users of the image:
RUN mkdir /workspace
WORKDIR /workspace

Last step is to prepare an entrypoint. That is a command that will be executed when the container starts. This command will start the Zowe CLI daemon so it is ready for our CLI commands. That is done by the script docker-entrypoint.sh:

Since we are using Secure Credential Store we need to do inside D-Bus session.

# Use entry point that initialized dbus and gnome-keyring-daemon for Zowe secure credential store and starts Zowe CLI daemon:
COPY ./docker-entrypoint.sh /zowe/
ENTRYPOINT [ "dbus-run-session", "--", "/zowe/docker-entrypoint.sh" ]
CMD ["zowe-init"]

You can find the complete Dockerfile here.

Using Docker Image on Your Computer

So, you have a Dockerfile but how do you use it?

First, you need to build it:

docker build -t zowe-cli .

Then you can start a container using it:

docker run --name=zowe-cli --cap-add ipc_lock --rm -it -d --mount type=bind,source="$PWD",target=/workspace,consistency=delegated zowe-cli

The container will be named zowe-cli and it mounts the current directory on your computer to the /workspace directory in the host. The--cap-add ipc_lock option is required for Secure Credential Store and -d makes the container to run on the background. The container is ready to execute your Zowe CLI and other commands:

docker exec zowe-cli zowe --version

Using Zowe CLI Team Configuration with the Container

In Zowe V2, you will have typically your project configuration for Zowe CLI stored in zowe.config.json and that will be in your Git repostiory. It does not contain the secure fields like user ID or password. To provide them we will have to run the zowe config secure command in the container:

docker exec -it zowe-cli zowe config secure
Enter profiles.base.properties.user - blank to skip: *** 
Enter profiles.base.properties.password - blank to skip: ***

Note that we are using -it options to run the command interactively with a terminal.

After that you can run any Zowe command that needs credentials — for example:

docker exec zowe-cli zowe zosmf check status

Using the Image On Jenkins

Publishing the Image

The first step is to publish the image to a registry with Docker images. In your company, you might be using Artifactory or a similar tool. You need to login to it:

docker login docker.artifactory.acme.net

And then you can tag the image that you have built and push it to the registry:

docker tag zowe-cli docker.artifactory.acme.net/zowe-cli:latest
docker push docker.artifactory.acme.net/zowe-cli:latest

After that, your image is ready to be used by everyone in your organization!

Using the Image in Jenkins

Let’s see an example and explain the major parts in it:

The agent section specifies what agent will be used. You can use a Docker container by using the docker section. Note that there is an additional argument --cap-add ipc_lock that is required for the Secure Credentials Store.

In this example we assume that zowe.config.json contains information about all the systems that you will need and share the same base profile with credentials. You just need to obtain credentials from Jenkins using the withCredentials declaration and then set them in the container using zowe config set command. If your configuration is more complex, you can use the same approach and set credentials to other profiles than the base one.

After that you can run any zowe command in Jenkins!

Summary

We learned about three major approaches how to run Zowe CLI on Jenkins. We did a deep dive into using Zowe CLI with Docker because that provides best value in long-term.

Photo by frank mckenna on Unsplash

If you enjoyed this blog checkout more Zowe blogs here. Or, ask a question and join the conversation on the Open Mainframe Project Slack Channel #Zowe-dev, #Zowe-user or #Zowe-onboarding. If this is your first time using the OMP slack channel register here.