Password management for Zowe CLI profiles

{Core} At a recent conference, a customer came up to me saying that although they loved Open Mainframe Project’s Zowe CLI and Zowe Explorer for VS Code, they ran into issues with passwords being revoked and how to manage password updates. For example:

• A number of Zowe profiles to different CLI extensions on the same LPAR share the same user ID and password. How can this be done in a way that the zowe.config.json profile definitions only store the credentials once ?
• When a new TSO password has been set outside of Zowe, what is the way to actively update the value used by the Zowe CLI and Zowe Explorer BEFORE the old expired value is used ? The user described situations when Zowe was repeatedly re-using his saved old (now invalid password) which resulted in a three strike user ID revoke occurring. Not good :-(

It wasn’t the first time I’d heard woes from a customer about Zowe CLI using an incorrect password, so with the help of some smart fellow Zoweians (Gene Johnson, Billie Jean Simons, Adam Wolfe and Jakub Balhar) I penned this blog to discover everything I could about CLI profiles, secure properties, and hopefully prevent any more users getting their credentials revoked with Zowe as the culprit.

Zowe .profiles versus zowe.config.json

Before we get started, this blog will only make sense and work for you if your CLI profiles are in a zowe.config.json file (which was introduced in Zowe v2 and in Zowe v3 will be the only way to persist profiles).

If you aren’t sure whether you have v1 or v2 profile persistence please read Gene Johnson’s excellent blog Enterprise rollout of Zowe CLI.

Sharing passwords between different profiles

Consider the scenario where an SSH profile and a z/OSMF profile are on the same LPAR. One way to describe this might be to have them each include their own set of properties as shown below:

{
  "$schema": "./zowe.schema.json",
  "profiles": {
    "tvt_zosmf": {
      "type": "zosmf",
      "properties": {
        "host": "zowehost",
        "port": 443,
        "rejectUnauthorized": false,
        "protocol": "https"
      },
      "secure": [
        "user",
        "password"
      ]
    },
    "tvt_ssh": {
      "type": "ssh",
      "properties": {
        "host": "zowehost",
        "port": 22,
      },
      "secure": [
        "user",
        "password"
      ]
    },
  },
  "defaults": {
    "zosmf": "tvt_zosmf",
    "ssh": "tvt_ssh",
  },
}

This works great, except that the "host", "user”and "password" attributes are defined twice, which means they need to be managed twice and aren’t shared. It also means that if the password changes you need to update in both places, and have more possibilities of this going wrong and a wrong stale password being used.

It’d be nice to have the common values captured only once. Especially because servers behind ports 22 and 443 are on the same LPAR so the userID and password are the same TSO credential.

There are two ways to solve the sharing problem: Base Profiles and Nested Profiles.

Base Profiles

As well as profiles that define endpoints you want to connect to for sending requests and receiving data (such as z/OSMF on 443 and SSH on 22 ) Zowe provides a property type "base" .

In the zowe.config.json below there is a profile "tvt_Base” defined with a "type" of "base" . This has the shared attributes that are common to both zosmf and ssh endpoints, namely the "host": "zowehost" (and the "rejectUnauthorized": false property the server is using a self signed certificate authority).

{
    "$schema": "./zowe.schema.json",
    "profiles": {
        "tvt_base": {
            "type": "base",
            "properties": {
                "host": "zowehost",
                "rejectUnauthorized": false
            },
            "secure": [
                "user",
                "password"
            ]
        },
        "tvt_zosmf": {
            "type": "zosmf",
            "baseProfile": "tvt_base",
            "properties": {
                "protocol": "https",
                "port": 443
            }
        },
        "tvt_ssh": {
            "type": "ssh",
            "baseProfile": "tvt_base",
            "properties": {
                "port": 22
            }
        }
    },
    "defaults": {
        "zosmf": "tvt_zosmf",
        "ssh": "tvt_ssh",
        "base": "tvt_base"
    },
    "autoStore": true
}

To show that the user and password are shared I did a little experiment, the results of which are in the animated GIF below

• Run zowe jobs list jobs which is a zosmf command (so uses "tvt_zosmf" . This needs a userID and password and none are available so the message "Enter the user name for the service (will be hidden): is shown.
• Cancel out (using Control C) so no value is entered or stored.
• Run zowe ssh issue command “pwd" which is a ssh comand (so uses "tvt_ssh" ). This again has no userID and password so the promot appears again.
• Cancel out again — we have proved that no user ID and password can be found for either command
• Re-run zowe jobs list jobs and enter the user name and password. These are stored and a list of my JES jobs shown.
• Re-run zowe ssh issue command "pwd" . This successfully shows my working directory WITHOUT prompting for a userID and password.

Because when the second ssh profile command was run it didn’t ask me for my password this proves to me (and you as well I hope) that the credentials were stored for the "tvt_zosmf" profile and shared with the "tvt_ssh" profile because they were declared once in the zowe.config.json file by virtue of the "baseProfile": "tvt_base" .

Nested Properties

Another approach to consider instead of "base" profiles is nested profiles where "profiles" can define child "profiles" .

{
  "$schema": "./zowe.schema.json",
  "profiles": {
   "tvt_root": {
     "properties": {
       "host": "zowehost"
     },
     "secure": [
       "user",
       "password"
     ],
     "profiles": {
       "zosmf": {
         "type": "zosmf",
         "properties": {
           "protocol": "https",
           "rejectUnauthorized": false,
           "port": 443
         }
       },
       "ssh": {
         "type": "ssh",
         "properties": {
           "port": 22
         }
       }
     }
   }
}

The nested version above has "profiles" on line 3 which defines parent (or outer) profile with the common "host": “zowehost" together with "secure" to capture the "user" and "password" . Beneath the parent profile are two child "profiles" for "zosmf" and "ssh" that define specific unique properties for each profile type.

Note: Because the profiles are nested, the name of the child profiles are parentProfileName.childProfileName. This means that the "zosmf" profile becomes tvt_root.zosmf and the "ssh" profile becomes tvt_root.ssh .

Let’s try the same trick we did with baseProfiles, which is to show that neither "tvt_root.zosmf" nor "tvt_root_ssh" has credentials, then we’ll provide them the second time round and show that once we’ve provided then to run the zosmf zowe jobs list jobs command they’re no longer requested by zowe ssh issue command "pwd" which successfully retrieved the working directory (in this case /u/winchj cause my user ID is WINCHJ ).

To Base or to Nest, that is the question ?

Personally I prefer nested profiles to base profiles. Rather than bore you with why, let’s crack on and talk about secure property storage (which is where password woes occur), and if you make it to the end of this blog I have a section that nerds out a bit on the base vs. nested debate.

Secure property values — do I have to ?

Tip: Don’t do this in production but if you want to store your credentials in-line and bypass the secure credentials store, you can do this by in-lining the "user" and "password” password directly in the "properties".

  "tvt_root": {
    "properties": {
      "host": "zowehost",
      "user": "MYUSERID",
      "password": "MYPASWRD"
    },
  },

The reason we ask you not to in-line the "password” property is because your TSO user ID and password within the ~/.zowe/zowe.config.json file are in the clear (shown above). Having your password in plain text in an unencrypted file leaves your PC vulnerable to malware or other bad actors trying to snarf the value.

The preferred and recomended approach is to always use "secure" properties for Zowe CLI profiles. However, in-lining the credentials is sometimes good for testing and can get you out of a tight corner. Such an example is where you are running into issues where the secure value is incorrect and you don’t know how to change it and want to provide a temporary in-line override.

Secure Credentials

When the Zowe CLI stores the password away for you it’s placed into the same area that your PC is using to store things like VPN client certificates and public keys of web server signing authorities. On my macbook I can access this using the Keychain Access application. On a Windows PC the equivalent application is Credential Manager found in the Control Panel.

In the screen animation below I’ve opened Keychain Access alongside a terminal window to show in real time that after the CLI prompts me for the user ID and password a Zowe item appears in the keystore. This is then re-used the second time I run the CLI command (it’s been stored and recalled) and if I want to tell my PC to kindly forget the password I can just delete it from the keystore and Zowe will re-prompt me and store the new one.

This is one way to get Zowe CLI to forget a password that you know is invalid, without the embarrassment of you throwing it at z/OSMF who will check it with SAF and might use up one of your 3 password challenges. It’s not the best way however as it will remove all of the secure credentials that Zowe has stored across all profiles, but it’s good to know it’s available and understand how the properties are actually persisted on your PC.

Re-prompt for all credentials

A great command to make your friend is zowe config secure . What this does is ask you to re-enter all properties that are defined in zowe.config.json as being stored securely. This is great if you don’t don’t fancy futzing around with your PC’s key management tool, and also has the advantage that it’s fine grained so you can update one property at a time whereas deleting your zowe entry in the keystore will forget every property value that Zowe has squirreled away there.

Living dangerously, the screen shot below shows that I deliberately put a bad password into the secure credentials store giving me a 401 error. To fix this I then ran zowe config secure with a --gc option for Global Configuration. The CLI recognizes the two properties in my zowe.config.json file that are held securely and prompts me. I skipped the user ID (as that hadn’t changed) and re-typed my new password which was stored securely, and re-ran the zowe jobs list jobs command to prove that the fresh password correctly got through to show the list of my jobs.

(For the eagle eyed who are wondering why I am no longer typing --zosmf-p tvt_root.zosmf after my CLI commands with nested profiles, it’s because I updated my zowe.config.json file to make them the defaults.

 "defaults": {
   "zosmf": "tvt_root.zosmf",
   "ssh": "tvt_root.ssh"
 },

Hint: One nice side effect of nested profiles, is that if you have a lot of z/OSMF profiles you don’t need to call them all names like zosmf_prod and zosmf_lparA and so forth, you can just call the paarent profile prodand lparA and call the child profiles can be called the sane as the type ( zosmf , ssh , cics , … ) so it becomes easier to manage as the name just becomes prod.ssh . I personally find this simpler to use and you likely will too !

Note: The command zowe config secure will ask for a property value based on the definition in zowe.config.json even if no value currently exists in the secure credentials store, making it a good way to initialize values. This is especially true in enterprise environments where someone has given you a copy of a zowe.config.json file with all of the host definitions, ports, and values that are common between users, and your finishing touch is just to provide your own userID and password so Zowe operates under your credentials (and not someone else’s).

When prompted for a secure property value with zowe config secure you can press Enter to skip any update, which allows you to cycle through all defined secure properties until the one you want to change appears and then change just that one on its own.

Scalpel property value changes

The final thing to make your new best friend is the command zowe config set . This allows you to precisely target a particular property value. In the gif above you can see that the console is echoing the value profiles.tvt_root.properties.password . This corresponds to the path location of where the password is held for the tvt_root profile. Whatever property you want to update you can find it by looking at the nesting level in the zowe.config.json file and turning this into a . delimited path.

... 
  "profiles": {
    "tvt_root": {
      "properties": {
        "host": "zowehost"
      },
      "secure": [
        "user",
        "password"
      ],
    },
...

In the screen shot below (to prove to me and you that it does work), I’m going to introduce you to --show-inputs-only that can be tagged onto the back of any Zowe CLI command to see all of the data that is being flowed to the REST API call without it actually being invoked.
To see the any properties retrieved from the secure properties the shell variable SHOW_SECURE_ARGS needs to be true (this is to protect against
accidentally echoing a password or other piece of sensitive data).

You’ll see that my user ID is WINCHJ and the password is BOB.
Although we love BOB it is not the correct password for my user ID and z/OSMF REST API will throw a 401 bad credentials error when I send it over the wire. The correct TSO password for WINCHJis BILL and I can change the one Zowe is using with zowe config set “profiles.tvt_root.properties.password” “BILL” — gc shown below.

(For anyone from IBM CISO security reading this, I can assure you that BILLisn’t my actual long term password and it was temporarily set for the purpose of illustrating this blogpost and no plants or animals were harmed during the recording of this animated GIF).

Hint: If you do need to change your actual z/OS TSO password (which I needed to create some of the test cases captured in the animated GIF files for BOB and BILL) the command PASSWORD PASSWORD(OLDPWD NEWPWD) is how to get this done.

You could use an ISPF command shell as shown above, or if you’re feeling very brave and want to go full Zowe, you could do a zowe tso issue command "PASSWORD PASSWORD(OLDPWD NEWPWD)" with a TSO profile defined (see Zowe CLI and TSO Commands), before following this up with a zowe config set to update your secure credentials password (as you’ve now invalidated the previously held value). If you do this my Jedi friend, may the force be with you and let me know if you succeed first time — I’ll treat to you a tea / coffee / scone if we’re ever in the same cafe as you deserve it !

Why I prefer nested profiles for shared property values

Thank you for reading this far ! At the start this blog we saw how both "base" properties as well as nested "profiles" can be used to structure a zowe.config.json file to enable common property values to be defined once. I promised an explanation as to my personal preference for using nested, which is because "base" profile types arent’ just used for grouping values but are the basis for token based Authentication (see Zowe CLI: Token Based Authentication).

What happens for me is that in my zowe.config.json I have some profiles on a shared LPAR that are wanting to go through APIML token based authentication to port 7554 (which acts as a reverse proxy to the endpoint using tokenBasedAuthentication), while others I want to connect directly with userID and password but on the same LPAR. I find it easier to keep the responsibilities separate so LPAR and TSO grouping association is done through nested profiles while APIML tokenBasedAuthentication grouping achieved with base profiles.

If you don’t have APIML in play with your CLI profiles however you prob won’t feel this pain and you can use "base" profiles for grouping, but at some point you’ll need token Auth alongside multi-factor auth which requires getting APIML into the mix. When that happens you’ll likely reading this blog sometime after November 2023 thinking “Joe was right, I should have listened to him back then”.

Summary

• The command zowe config list can be used to see all values held in SCS.
• The command zowe config set can be used to change values held in SCS and press Enter to skip any you want to leave alone.
• The CLI value --show-inputs-only can be used to echo all values that will be sent on a command, paired with SHOW_SECURE_ARGS=true to echo any SCS values.
• The profile type "base" can be used to define shared properties between profiles, which point to the common root using "baseProfile" .
• Nesting "profiles" in the zowe.config.json file allows inner "profiles"to share common details such as "host" , "user" , and "password" with the outer "profile". This is a great way to define and store values only once in the configuration file, which makes the management of updates to these values (especially passwords) easier.
• If you’re in a bind consider in-lining properties into your profile’s "properties" , and also consider deleting entirely your zowe keychain value to get Zowe CLI to re-prompt and store fresh values de novo.

If you want to find out more about profiles PLEASE checkout Gene Johnson’s great blogpost Enterprise rollout of Zowe CLI. Another CLI gem is Dan Kelosky’s blog about storing CLI arguments in profiles that you’ll wonder how you lived without.

Find out more

If you enjoyed this blog checkout more Zowe blogs here. Or, ask a question and join the conversation on the Open Mainframe Project Slack Channel #Zowe-cli, #Zowe-user or #Zowe-onboarding. If this is your first time using the OMP slack channel register here.

Zowe™, and the Zowe™ logo, and the Open Mainframe Project™are trademarks of the Linux Foundation.”