Zowe CLI and client certificates

Published in

Zowe

11 min readJul 15, 2024

{Core} Client certificates are a great way to configure a PC to be able to issue Zowe CLI commands removing the need for the the PC to have knowledge of a user ID and its TSO password.

Theft of passwords and subsequent use as a login attack vector is the common way systems are “hacked”, so replacing them with a personal signed certificate (aka x509) is a more secure way to permit the client machine to run tasks under that user’s privileges. The client certificate approach also allows you to simultaneously lockdown traditional login access, providing an extra layer of security.

This blog shows the steps to create a certificate on z/OS for a TSO user ID and configure a Zowe CLI zosmfprofile to use that certificate for client authentication. My thanks to Adam Wolfe and Andrew Hahn from the Zowe CLI squad, as well as Mark Ackert from the Zowe systems squad (who provided the ACF2 and TopSecret commands), for their help in preparing this blog.

The process breaks down into three main steps:

- Create the certificate on z/OS associated with a TSO user ID
- Transfer the certificate to a desktop PC
- (Optional — test the certificate with z/OSMF using a web browser)
- Configure a CLI profile to use the certificate

Create a certificate on z/OS for a user ID

The z/OS security manager can create a certificate for us and associate that with our user ID (we’re WINCHJ for the purpose of this blog). All certificates need to be signed by a certificate authority (aka CA or CERTAUTH), and to access z/OSMF REST APIs and it’s a good idea to use the same CA as z/OSMF’s server certificate (If we did use another CA we’d have to then connect that to the z/OSMF keystore to ensure it’s trusted). The label of z/OSMF’s certificate will likely be zOSMFCA, however we can verify this by inspecting the keyring for the z/OSMF user ID of IZUSVR using the TSO command RACDCERT LISTRING(*) ID(IZUSVR) . I’ve used RACF samples throughout this blogpost however at the end you’ll find the equivalents for TopSecret and ACF2 which are security managers for z/OS from Broadcom.

Having run LISTRING if there is only one keyring listed this will be the one z/OSMF is using, and we can see below it’s called zOSMFCA on my system. If there are multiple keyrings you’ll need to look at the IZUSVR1 address space to see which one it’s using, or else dig into the z/OSMF configuration files. It’s rare there is more than one keyring and almost every time the CA label will be zOSMFCA, but given that this value is critical to get right before proceeding it’s always worth checking.

Because I’m a bit of a mainframe modernization nerd, that’s the last 3270 panel you’ll see me include in this blog. Henceforth I’m going to use zowe tso issue command <CMD>to run z/OS commands, adding — ssm which suppresses the TSO startup message being echo’d back in the response. For more info see Zowe CLIand TSO commands.

Now we know the label of the z/OSMF certificate authority, the next step is to create a personal certificate for the user ID we want to give the certificate to, in my case WINCHJ (aka me !). We can call this certificate whatever label or common name we want, but to help with keeping track of things on my PC I like to include the user ID plus a host short identifier that lets me remember which z/OS host I created the certificate for.

**Note** In this blogpost I chose tvt11 which is the LPAR mnemonic of the system I used for testing and creating the screen captures, however if you’re following along at home choose something appropriate for your z/OS host like maybe lpar_prod or test_lpar,,… .

Because we’re going to use the Zowe CLI with a direct connect to z/OSMF we’ll sign the certificate with the z/OSMF certificate authority which has a label of zOSMFCA .

RACDCERT ID(WINCHJ) GENCERT SUBJECTSDN(CN('User WINCHJ on tvt11') O('IBM') OU('Zowe') C('UK')) WITHLABEL('User WINCHJ on tvt11') SIGNWITH(CERTAUTH LABEL('zOSMFCA'))

After creating a certificate we’ll get a message telling asking to run SETROPTS RACLIST(DIGTCERT) REFRESH . This refreshes the live in-memory certificates from their definitions, after which we can run RACDCERT LIST ID(WINCHJ) to see our certificate details we just created . Let’s do all of this using the Zowe CLI with TSO commands.

Nicely done ! We now have a personal certificate that we created on z/OS for the user ID WINCHJ . The next step is to transfer this to the PC and teach the Zowe CLI profile to use it in place of a hard coded userID and password.

**Note** Because we are using the same CA that signed the z/OSMF certificate this will be trusted, however if we hadn’t stacked the deck and used a different CA chain we would have needed to add the TRUST argument to the end of the RACDCERT GENCERTcommand during generation, or if repair is needed post creation you can run RACDCERT ID(WINCHJ) ALTER(LABEL(‘User WINCHJ on tvt11’)) TRUST after followed by another SETROPTS RACLIST(DIGTCERT) REFRESH command. More detail on certificate trust is available at https://www.ibm.com/docs/en/zos/2.4.0?topic=certificates-racdcert-add-add-certificate. We’ll look more at the whole issue of trust in a follow up blogpost where we’ll be importing an externally generated client cert into z/OS as well as using a Zowe APIML client cert signed by the Zowe CA. More to come, but for now focus on the fact that we’re connecting to z/OSMF so our client cert it signed by the same CA that signed the z/OSMF server cert which means z/OSMF trusts it the same way families trust each other !

Get the certificate from z/OS to your PC

To transfer the personal certificate to the PC is a two part exercise as we can’t download it directly from RACF. Instead we must:

Export the certificate from RACF to a sequential data set
Transfer the certificate from the above data set to a local file on the PC

To do the export we’ll use the command RACDCERT EXPORT and point at the label for the certificate we created for the user above (which is User WINCHJ on tvt11), specify the certificate format we want to create which is PKCS12DER , and a dummy password of ‘password’.

RACDCERT ID(WINCHJ) EXPORT(LABEL('User WINCHJ on tvt11')) DSN('WINCHJ.CLIENT.P12') FORMAT(PKCS12DER) PASSWORD('password')

After creating the sequential data set WINCHJ.CLIENT.P12 with the exported z/OS certificate, this needs transferring this to our PC. I like to use the Zowe CLI to do this, and the following command zowe files download ds "WINCHJ.CLIENT.P12" -b -f winchj_tvt11.12 will download the certificate to a local file winchj_tvt11.12 . Note the -b to leave the file in binary format, so if you do choose to use ftp or another way to transfer the file please ensure it comes across without any codepage conversion.

Why do I need to specify a password for the .p12 keystore ?

We need to specify a password (which we can leave as ‘password’) because the PKCS12 DER file contains both a public certificate and the private key used to sign it. Keystores with private keys enforce passwords in case someone gets hold of the file, extracts the private key and uses it for malicious purposes.

I talk to some customers who get concerned that the private key in the PKCS12DER file is the same one used to encrypt data by z/OSMF, so that if someone got their hands on it together with data packets they could decrypt server data and look at information not intended for their eyes, or perhaps they could use the private key to impersonate another user ID and elevate their priviledges. This is not the case, the private key here is just one that was created by the original RACDCERT GENCERT command and every user in your organization that you create a z/OS client certificate for will have unique private keys for their personal certificate. This is not the private key used by z/OSMF to encrypt its server data.

Verifying the .p12 file

If you try and open the winchj_tvt11.p12 file you’ll see it’s a bunch of unreadable characters. To verify that the file was created and transferred correctly use the keytool command to list its contents. keytool -v -list -keystore winchj_tvt11.p12will display the key entries so you can confirm there is one PrivateKeyEntry together with its attribute for creation date, name and more.

Verifying the certificate works for z/OSMF login without Zowe

You can skip this step and go straight to the next section Split the .p12 file into the certificate and the private key, however I find it’s always good to verify that the certificate lets you log into z/OSMF’s user interface without getting the panel asking for a userID and password.

The z/OSMF system we’re working with is on the host name zowehost_11 and port 443 and its hompage is https://zowehost_11:443/zosmf . We haven’t configured our PC yet to use our certificate so we should expect to the login page prompting for the user ID and password as below.

To get our web browser to work with the certificate it needs to be imported into our PC’s keystore. For a macbook the command to access the keychain holding certificates this is KeyChain Access . Use menu File->Import and select the downloaded winchj_tvt11.p12 file. In the animated gif below I’ve selected a filter of tvt11 in the Keychain Access window top right corner to restrict the items shown and illustrate that after import the single .p12 file is split into two entries. The first is the certificate itself and the other is its private key.

To verify that we can use our cert to bypass the userID and password screen go back to its URL (in our case https://zowehost_11:443/zosmf ). What occurs is that when the browser does the TLS handshake the certificate we placed in our keystore will be recognized by z/OSMF, and we’ll be promoted for whether to use it or not. On the prompt we can view the certificate information (as shown below) just to dig around a bit, before selecting OK which will then perform a mutual TLS handshake (ala mTLS) and bring us to the z/OSMF homepage WITHOUT the need for a user ID and password screen.

So far so good. We have got a PKCS12 file with the certificate on our PC that we’ve succesfully used to get into z/OSMF without the need for a userID or password anywhere in sight.

Back to the main movie plot, which is to configure the Zowe CLI profile to do the same as the clever browser. We’re ready for the final act !

Split the .p12 file into the certificate and the private key

Just as the KeyChain Access import split the .p12 file into two parts; the private key and the client certificate, we’ll need to do this ourselves using some openssl commands.

For the private key we’ll extract it from the .p12 keystore into a file with a .key suffix, where it’ll be used for rsa public/private key encryption. The command to use is openssl pkcs12 -in winchj_tvt11.p12 -nocerts -nodes -out winchj_tvt11.key .

As well as the private rsa key, we also need the x509 client certificate itself. For this use openssl again but specify -clcerts -nokeys so the full command becomes openssl pkcs12 -in winchj_tvt11.p12 -clcerts -nokeys -out winchj_tvt11.crt .

Great ! Now we have the .key and the .crt file we can use this in our Zowe CLI profile. Remove the user and password and specify the full path name to the .crt file in the certFile property, and the full path name to the .key file in the certKeyFile property.

        "zosmf_svl_11": {
            "type": "zosmf",
            "properties": {
                "host": "zowehost_11",
                "port": 443,
                "rejectUnauthorized": false,
                "protocol": "https",
                "certFile": "/Users/joe/zowe/client_cert/winchj_tvt11.crt",
                "certKeyFile": "/Users/joe/zowe/client_cert/winchj_tvt11.key"
            },
            "secure": []
        },

To verify this works use a CLI command to make sure you don’t get any errors, and also it’s a good idea to add the --show-inputs-only parameter to double check you’re not flowing a userID and password across the wire but instead using your swish bang newly created and configured certificate !

In this blog we created our certificate on z/OS and downloaded to the PC, so it’s a bottom up scenario. In a follow up blog I’ll show how to use client certificates top down, i.e. take one that is created off platform by an external signing authority, upload that to z/OS and associate it with a TSO user ID, and configure a CLI profile.

Enjoy !

What if I’m using ACF/2 ?

In the TSO commands for this blog I’ve used RACF, however for customers who are using ACF/2 the equivalents are:

Listing the certificate authority for z/OSMF’s keyring

SET PROFILE(USER) DIVISION(KEYRING)
list like(IZUSVR.-) 

KEYRING / IZUSVR.KEYR01 LAST CHANGED BY ZOWEAD6 ON 01/22/20-13:43           
                     DEFAULT(IZUSVR.CERT01) RINGNAME(IZUKeyring.IZUDFLT)    
The following certificates are connected to this key ring:                  
CERTDATA record    Label                             Usage                  
-----------------  --------------------------------  --------               
CERTAUTH.ZOSMFSRV  zOSMFCA                           CERTAUTH               
IZUSVR.CERT01      DefaultzOSMFCert.IZUDFLT          PERSONAL

This shows that the Label is zOSMFCA however the CERTDATA record is CERTAUTH.ZOSMFSRV which is the string we need to use to sign the personal certificate.

Creating a personal certificate for a TSO user ID

You can set the label, cetficiate name and other attributes to something that lets you remember who the user ID is and the system name (which just helps when you’re looking at the cert on your PC and trying to remember its purpose), I’ve used User WINCHJ on tvt11 as that is my user ID and my LPAR mnemonic.

SET PROFILE(USER) DIVISION(CERTDATA)
followed by

GENCERT WINCHJ.ZOWECERT LABEL(User WINCHJ on tvt11) SUBJDSN(CN='User WINCHJ on tvt11) OU='Zowe' O='IBM' C='UK') KEYUSAGE(HANDSHAKE) SIGNWITH(CERTAUTH.ZOSMFSRV)

The GENCERT <USER>.ZOWECERT associates the certificate with the user specified in the HLQ.

Exporting the certificate to a data set

EXPORT WINCHJ.ZOWECERT DS(`WINCHJ.CLIENT.P12`) FORMAT(PKDS12DER) Password(password) .

Once the certificate is in the data set download with ftp in binary mode or else zowe files download ds as described earlier in the section Get the certificate from z/OS to your PC

What if I’m using TopSecret

Listing the certificate authority for z/OSMF’s keyring

tss list(izusvr) keyring(all)

which will list the keyring’s details, showing the CA’s label of zOSMFCA below.

KEYRING =  IZUSVRKR         ACCESSORID = IZUSVR                    
KEYRING LABEL = IZUKeyring.IZUDFLT                                 
KEYRING HAS THE FOLLOWING CERTIFICATES CONNECTED:                  
  ACID(CERTAUTH)  DIGICERT(ZOSMFCA )  DEFAULT(NO )  USAGE(CERTAUTH)
  LABLCERT(zOSMFCA                         )                       
  ACID(IZUSVR  )  DIGICERT(DEFOSMFC)  DEFAULT(YES)  USAGE(PERSONAL)
  LABLCERT(DefaultzOSMFCert.IZUDFLT

Creating a personal certificate for a TSO user ID

TSS GENCERT() DIGICERT(ZOWECERT) SUBJECTTN('CD="User WINCHJ on tvt11" O="Zowe" C="UK") LABLCERT('WINCHJ CERTIFICATE') KEYUSAGE(HANDSHAKE) SIGNWITH(CERTAUTH,ZOSMFCA)

The association of this certificate with the userID is because of the LABLCERT specifying <userid> CERTIFICATE which is echo’d back by TSS

TSS0300I GENCERT FUNCTION SUCCESSFUL
TSS1624I CERTIFICATE ZOWECERT HAS BEEN ADDED TO USER WINCHJ

You can also check that the userID has a certificate associated with it by running the command

tss list(WINCHJ) digicert(ZOWECERT) .