Zowe
Published in

Zowe

Keep certificates stored in the SAF keyrings

Certificate ring in ESM (Key Ring by Denise Krebs)

{Core} By now, you’ve probably heard from your colleagues, peers, and in conference presentations about the many advantages of using Open Mainframe Project’s Zowe to leverage API services. Perhaps you’ve experienced first-hand how Zowe CLI tools can simplify CI/CD implementation, or how the Virtual Desktop helps to provide new developers without TSO experience access to the MF environment, or even how Single Sign-On and the API Mediation Layer Catalog can streamline API workflow. Perhaps you’ve read the documentation, and everything sounds good, the configuration seems simple enough, and then you encounter an unexpected bump in the road — the certificates for the SSL need to be exported to file-based key stores and held in Unix System Services (USS) in Keystore format. At least that’s how it used to work for Zowe until the 1.15 release that was made available in September 2020 — available on the Zowe download site.

To start off, let’s do a short recap on what SAF keyrings are. According to the IBM Documentation: “A keyring is a collection of certificates that identify a networking trust relationship (also called a trust policy). In a client-server network environment, entities identify themselves using digital certificates.” The private keys for the certificates need to be treated in the most secure way possible and ideally not distributed. With the SAF Keyrings, it is possible to generate certificates directly into the keyring without the private key ever leaving the SAF mainframe security. Before this was included with Zowe the certificate needed to be copied to a USS Keystore file which, while still under mainframe security access, was an additional step that system programmers had to manage, control, and audit.

So what does this mean in terms of providing better security in Zowe?

The Zowe team is well aware of the additional security measures that Enterprise Security Managers (ESMs) provide — storing certificates as an alternative to other ways of securing file-based key stores that have traditionally been used by Java applications off the MainFrame. With this in mind, Zowe dev teams have been working on providing security support to all components that make up Zowe.

Zowe release 1.15 supports the use of certificates stored within traditional SAF keyrings.

You may be wondering, “should I be concerned with this support if I already use the older Zowe versions, and I am ok with the way the things currently are?” Simply put, there’s no reason for concern. Support of certificates stored in file-based Java trust stores stays with us.

“Ok, but what if I’m running an application using enablers (i.e. Java libraries that do the heavy lifting for the application) to connect to the API Mediation Layer within Zowe… can I also use the SAF keyrings?”

Sure, conformant applications onboarded to the API Mediation Layer may use the enablers as long as your application has proper access to z/OS and the keyrings. The section “Keyrings in applications extending API Mediation Layer” presented later in this article covers this topic in more detail.

Zowe Installation

Let’s say you’re installing Zowe and voila, you’re in, and now that you finally don’t have to expose private certificates, you’re ready to give Zowe a try. So how are you going to set it up? The Zowe team (special kudos here to Vit Tomica) prepared guidelines in the documentation outlining the paths towards this setup. We’ll briefly touch on the key parts here, but if you want to look at this in more detail go to Zowe Docs.

The default path which you should follow if you want to install ZOWE for production is to customize and submit the ZWEKRING JCL, and finalize the certificate configuration by executing the zowe-setup-certificate.sh script. If you have RACF installed as ESM on your system, you can use just the zowe-setup-certificate.sh script for evaluation purposes.

Customize and run ZWEKRING

The ZWEKRING JCL offers 3 options of certificate configuration in a keyring:

  • Use existing certificates in a security product
  • Generate a new certificate using local Zowe CA
  • Import an existing certificate from a PKCS12 data set to the security product

Let’s see an example for each option.

If you’re using RACF to generate a new certificate, do the following:

1) Make sure the PRODUCT symbol is set to the RACF value

//         SET  PRODUCT=RACF         * RACF, ACF2, or TSS

2) Set HOSTNAME and IPADDRESS symbols. They are used as Subject Alternative Names for the certificate

//*      * Hostname of the system where Zowe is to run
// SET HOSTNAME='www.example.com'
//* * IP address of the system where Zowe is to run
// SET IPADDRES='192.168.1.1'

3) Set the ROOTZFCA symbol so that it matches a name/label of the root CA of the z/OSMF certificate on your system. This step is required to set up trust between z/OSMF and Zowe.

//*      * Name/Label of the root CA of the z/OSMF certificate
// SET ROOTZFCA=zOSMFCA

4) You may modify other symbols according to your needs.

5) Submit the JCL and review the output.

6) Edit the zowe-setup-certificates.env file so that relevant environment variables in the file match the symbols in the JCL.

# Select an alias for the certificate in the generated keystore.
# If you used the ZWEKRING jcl, then this variable has to be set to the
# Zowe certificate's LABEL specified in the JCL.
KEYSTORE_ALIAS=localhost
# Specify zowe user id to set up ownership of the generated certificates.
# This variable is also used for keyring configuration. If you used
# the ZWEKRING jcl, set the variable to the same user id as in the jcl.
ZOWE_USER_ID=ZWESVUSR
# Specify zowe keyring that keeps zowe certificates, if not specified then
# USS keystore files will be created. If you used the ZWEKRING jcl, set
# the variable to the same keyring that you used in the jcl.
ZOWE_KEYRING=ZoweKeyring
# If you used ZWEKRING jcl to configure the certificates for the keyring
# then set this variable to false (defaults to false)
GENERATE_CERTS_FOR_KEYRING=false

7) Execute the zowe-setup-certificates.sh script with the zowe-setup-certificates.env script as an input file.

If everything went fine, you should be all set and ready to run Zowe on a keyring.

If you’re using TopSecret to use an existing certificate, do the following:

1) Make sure the PRODUCT symbol is set to the TSS value

//         SET  PRODUCT=TSS         * RACF, ACF2, or TSS

2) If the existing certificate is signed by an external CA, be sure to connect all certificates from the signing CA chain to the keyring. For example, if the CA chain consists of one root CA (e.g. ROOTCA) and one intermediate CA (e.g. INTERCA1), then set the following symbols:

//*      * Name/Label of the intermediate CA of the Zowe certificate
//* * Ignore if not applicable
// SET ITRMZWCA=INTERCA1
//* * Name/Label of the root CA of the Zowe certificate
//* * Ignore if not applicable
// SET ROOTZWCA=ROOTCA

All CAs in the signing chain should be owned by the CERTAUTH acid. If there is more than one intermediate CA in the signing chain, then be sure to insert an appropriate number of TSS command(s) that connect the CAs to the keyring. For example,

TSS ADD(&ZOWEUSER.) KEYRING(ZOWERING) LABLRING(&ZOWERING.) +
RINGDATA(CERTAUTH,INTERCA2) USAGE(CERTAUTH)

3) Set the ROOTZFCA symbol so it matches the name/label of the root CA of the z/OSMF certificate on your system. This step is required to establish trust between z/OSMF and Zowe.

//*      * Name/Label of the root CA of the z/OSMF certificate
// SET ROOTZFCA=zOSMFCA

4) Comment out the “Option 2” block according to the instructions in the JCL.

/* Option 2 - BEGINNING ............................................ */
/* Create a certificate signed by local zowe's CA .................. */
/*TSS GENCERT(&ZOWEUSER.) +
/* DIGICERT(ZOWECERT) +
/* SUBJECTN( +
/* 'CN="&CN. certificate" +
/* OU="&OU." +
/* O="&O." +
/* L="&L." +
/* SP="&SP." +
/* C="&C." ') +
/* KEYSIZE(2048) +
/* NADATE(05/01/30) +
/* LABLCERT(&LABEL.) +
/* KEYUSAGE('HANDSHAKE') +
/* ALTNAME('DOMAIN=&HOSTNAME') +
/* SIGNWITH(CERTAUTH,ZOWECA)
/* Option 2 - END .................................................. */

5) Modify a TSS command that connects the existing certificate to the keyring. For example, if your existing certificate is owned by the special SITE acid, and the DIGICERT name of the certificate is CERT1234,then customize the TSS command as shown here:

/* Connect a Zowe's certificate with the keyring ................... */
TSS ADD(&ZOWEUSER.) KEYRING(ZOWERING) LABLRING(&ZOWERING.) +
RINGDATA(CERTSITE,CERT1234) USAGE(PERSONAL) DEFAULT

6) If the existing certificate is owned by the SITE acid, you have to uncomment the following command in the JCL. This will allow the Zowe userid to use a private key from the SITE owned certificate.

/* Uncomment this command if SITE acid owns the Zowe certificate   */
TSS PERMIT(&ZOWEUSER.) IBMFAC(IRR.DIGTCERT.GENCERT) ACCESS(CONTROL)

7) Submit the JCL and review the output.

8) Edit the zowe-setup-certificates.env file so that relevant environment variables in the file match the symbols in the JCL.

# Select an alias for the certificate in the generated keystore.
# If you used the ZWEKRING jcl, then this variable has to be set to the
# Zowe certificate's LABEL specified in the JCL.
KEYSTORE_ALIAS=localhost
# Specify zowe user id to set up ownership of the generated certificates.
# This variable is also used for keyring configuration. If you used
# the ZWEKRING jcl, set the variable to the same user id as in the jcl.
ZOWE_USER_ID=ZWESVUSR
# Specify zowe keyring that keeps zowe certificates, if not specified then
# USS keystore files will be created. If you used the ZWEKRING jcl, set
# the variable to the same keyring that you used in the jcl.
ZOWE_KEYRING=ZoweKeyring
# If you used ZWEKRING jcl to configure the certificates for the keyring
# then set this variable to false (defaults to false)
GENERATE_CERTS_FOR_KEYRING=false

9) Execute the zowe-setup-certificates.sh script with the zowe-setup-certificates.env script as an input file.

If everything went fine, you should be all set and ready to run Zowe on a keyring.

If you’re using ACF2 to import a certificate from a dataset, do the following:

1) Make sure the PRODUCT symbol is set to the ACF2 value

//         SET  PRODUCT=ACF2         * RACF, ACF2, or TSS

2) Set the DSNAME symbol to a data set name that contains the PKCS12 package with the certificate. Set the PKCSPASS symbol that contains the password of the PKCS12 package.

//*      * Name of the data set containing Zowe's certificate (PKCS12)
// SET DSNAME=HLQ.ZOWE.P12
//* * Password for the PKCS12 data set
// SET PKCSPASS='password'

3) If the imported certificate is signed by an external CA, then the PKCS12 data set should contain a complete signing CA chain. You have to connect all certificates from the signing CA chain to the keyring, make sure that all CAs in the signing chain are owned by the CERTAUTH acid. For example, if the CA chain consists of one root CA (e.g. ROOTCA) and one intermediate CA (e.g. INTERCA1), then set the following symbols:

//*      * Name/Label of the intermediate CA of the Zowe certificate
//* * Ignore if not applicable
// SET ITRMZWCA=INTERCA1
//* * Name/Label of the root CA of the Zowe certificate
//* * Ignore if not applicable
// SET ROOTZWCA=ROOTCA

If there is more than one intermediate CA in the signing chain, then you have to insert an appropriate number of ACF2 command(s) that connect the CA’s to the keyring. For example,

SET PROFILE(USER) DIVISION(CERTDATA)
CONNECT CERTDATA(CERTAUTH.INTERCA2) RINGNAME(&ZOWERING.) +
KEYRING(&ZOWEUSER..ZOWERING) USAGE(CERTAUTH)

4) Set the ROOTZFCA symbol so that it matches a name/label of the root CA of the z/OSMF certificate on your system. This step is required to set up trust between z/OSMF and Zowe.

//*      * Name/Label of the root CA of the z/OSMF certificate
// SET ROOTZFCA=zOSMFCA

5) Comment out the “Option 2” block according to the instructions in the JCL.

* Option 2 - BEGINNING ............................................ */
* Create a certificate signed by local zowe's CA .................. */
* SET PROFILE(USER) DIV(CERTDATA)
* GENCERT &ZOWEUSER..ZOWECERT +
* SUBJSDN(CN='&CN. certificate' +
* OU='&OU.' +
* O='&O.' +
* L='&L.' +
* SP='&SP.' +
* C='&C.') +
* SIZE(2048) +
* EXPIRE(05/01/30) +
* LABEL(&LABEL.) +
* KEYUSAGE(HANDSHAKE) +
* ALTNAME(IP=&IPADDRES DOMAIN=&HOSTNAME) +
* SIGNWITH(CERTAUTH.ZOWECA)
*
* Option 2 - END ................................................... */

6) Uncomment the “Option 1” block according to the instructions in the JCL.

* Option 1 - BEGINNING ............................................ */
* Import external certificate from data set ....................... */
SET PROFILE(USER) DIV(CERTDATA)
INSERT &ZOWEUSER..ZOWECERT +
DSNAME('&DSNAME.') +
LABEL(&LABEL.) +
PASSWORD('&PKCSPASS.') +
TRUST

* Option 1 - END .................................................. */

7) Submit the JCL and review the output.

8) Edit the zowe-setup-certificates.env file so that relevant environment variables in the file match the symbols in the JCL.

# Select an alias for the certificate in the generated keystore.
# If you used the ZWEKRING jcl, then this variable has to be set to the
# Zowe certificate's LABEL specified in the JCL.
KEYSTORE_ALIAS=localhost
# Specify zowe user id to set up ownership of the generated certificates.
# This variable is also used for keyring configuration. If you used
# the ZWEKRING jcl, set the variable to the same user id as in the jcl.
ZOWE_USER_ID=ZWESVUSR
# Specify zowe keyring that keeps zowe certificates, if not specified then
# USS keystore files will be created. If you used the ZWEKRING jcl, set
# the variable to the same keyring that you used in the jcl.
ZOWE_KEYRING=ZoweKeyring
# If you used ZWEKRING jcl to configure the certificates for the keyring
# then set this variable to false (defaults to false)
GENERATE_CERTS_FOR_KEYRING=false

9) Execute the zowe-setup-certificates.sh script with the zowe-setup-certificates.env script as an input file.

If everything went fine, you should be all set and ready to run Zowe on a keyring.

Can I just run the script?

Sure. Apart from the previous solution built upon the ZWEKRING JCL, there is another solution that works for RACF and allows you to start the ZOWE from keyring . This option isn’t recommended for production usage but if you are just trying Zowe, then it can simplify your first installation.

In order for the script to work, it is necessary to provide to the following Zowe environment configuration properties with valid values:

  • ZOWE_KEYRING= — set this variable to the keyring name
  • GENERATE_CERTS_FOR_KEYRING=true — set this variable to true

Keyrings in applications extending API Mediation Layer

Until now, If you are using the enablers provided by the API Mediation Layer to set up proper communication between your service and API Mediation Layer, you had to export the certificates to the file-based stores and then change them and move them back to the safety of the SAF keyrings.

To do this, you previously had to run the application on the z/OS in order to access the SAF keyrings and needed Java support to be available. You also had to provide a valid handler for the keyrings key store which was achieved by starting the Java application with the parameter -Djava.protocol.handler.pkgs=com.ibm.crypto.provider.

Now, the only thing which needs to change if you are already running the application on the platform is the type and path to the keystore and truststore. The keystore and trustore type needs to be set as JCERACFKS. This tells Java to use the keyring related classes and look for the path that leads to the specific keyring provided within. It looks like this:

ssl:
keyStore: safkeyring:////my_racf_id/my_key_ring
keyStoreType: JCERACFKS
trustStore: safkeyring:////my_racf_id/my_key_ring
trustStoreType: JCERACFKS

Conclusion

On the way to providing open and secure access to the mainframe, the Zowe team has made another step, perhaps more accurately a significant leap, by providing not only Single-Sign-On functionality but in addition making sure that the certificates can stay safely stored within the SAF without any need to expose them to the hostile outside world.

If you enjoyed this blog checkout more Zowe blogs here. Or, ask a question and join the conversation on the Open Mainframe Project Slack Channel #Zowe-dev, #Zowe-user or #Zowe-onboarding. If this is your first time using the OMP slack channel register here.

--

--

--

Zowe is the modern open source interface for the mainframe and z/OS. The Zowe blog has how-to’s, tips & tricks and other helpful insights for mainframe devops. Zowe is a project within the Linux Foundation’s Open Mainframe Project (OMP). Download @ Zowe.org.

Recommended from Medium

CS 373 Fall 2021: Srikar Ganti: Final Entry

How to learn Golang

Proprietary LIMS can thrive by embracing Commercial Open Source

What if there is a Tinder-like app for Music Streaming?

Median Finding Algorithm

Rest Assured : Serialisation and De-Serialisation

Blog | Java vs Kotlin — which one will prevail?

Setting up Ruby on Rails Project

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jakub Balhar

Jakub Balhar

I always try to find answers to the complex questions. Now exploring the world of Mainframes in the Broadcom inc.

More from Medium

Tips: Running z/OS REXX from Zowe CLI (Faster)

Developing Containers on zCX

A matrix of shipping containers.

Shift-left DevOps Database Operations With Db2 Schema Provisioning

Azure Sphere