Automatically Secure Code-Signing Assets

Problem

As I wrote previously in “Bring Customers Joy With Automation”, we were faced with the challenge of safely securing the customer’s signing assets in an automated way.

At this stage, we still had to manually process the data provided from the Appian Provisioner. We wanted to improve our process to make sure that there was no chance that we could see the passwords for our customers’ keychains.

Remember, the keychain contains a certificate and a private key that can be used to code-sign a mobile application to prove to users that the application was made by the company and was not modified by someone who may want to harm the company or its users. The Appian Provisioner application encrypts this data before it is ever sent to us.

We needed to secure the data sent to us from the Appian Provisioner in such a way that Appian employees were only allowed to store the customer’s data without being able to read it. We also had to make sure that only the build machines could read and decrypt the customer’s credentials in order to code-sign their custom mobile applications.

Solution

Appian uses Hashicorp Vault to store this kind of data. Vault is a server that secures, stores, and tightly controls access to tokens, passwords, certificates, and encryption keys for protecting secrets and other sensitive data using a website UI, command line interface, or HTTP API endpoints.

To ensure that each entity had the correct permissions, we decided to use Vault’s policies and token roles. A Vault policy allows you to configure authorization access to create, update, read, list, and delete secrets. Policies can be associated with users or groups. A Vault token role allows you to provide a token to another user to take on a role that uses a given policy.

For example, there may exist a token role that is tied to a policy that allows a token-holder to read a particular secret. I may not be able to read that secret, but I could create a token that allows you to login on your computer and read that secret without allowing me to do the same.

So, that’s what we did. We created two distinct policies: one that allowed us to write — and only write — customer data, and the other that allowed an automated system to read — and only read — customer data. We also secured the method of decrypting the customer’s passwords.

I’ll guide you on how to do each of these things so that you too can use this Vault functionality to secure your own sensitive data.

Vault Setup

To get the most out of this article, I suggest that you set up an instance of Vault as explained in the article Docker compose — Hashicorp’s Vault and Consul Part A. For this example, you can stop after you’ve unsealed Vault (before auditing is enabled).

Once you set up your own debug instance of Vault, if you haven’t already, run this command inside of your Vault container:

vault kv enable-versioning secret

This will allow you to store multiple versions of secrets. If you make a mistake, you can simply revert back to an older version of the secret.

Encryption Keys

As I wrote above, the Appian Provisioner encrypts the customer’s keychain password and APNS key, and we needed our build system to decrypt this data. To do this, we used a public and private key pair to encrypt and decrypt the data.

I used the article Encrypt and decrypt files to public keys via the OpenSSL Command Line as a reference to create the following script that creates a OpenSSL private key and public key in order to perform Public-key encryption.

This writes the private key its passphrase to the Vault path: secret/data/custom-mobile-apps/crypto. If you followed the Docker compose article I referenced above, you can run this script from the Vault-Consul-Docker/ directory and use the Initial Root Token for the purpose of debugging (each Admin user for Vault should have their own tokens and one shouldn’t normally use the Root Token).

Allow Users to Only Write Customer Data to Vault

We want to secure the customer’s credentials in Vault, but that means that we have to first allow the human user the ability to write into Vault.

Let’s create Vault policies that allow that by defining it:

Users with this policy are allowed to write keychain and APNS key data to Vault with the specified paths.

Now that we have defined the policy, we have to write it into Vault. From the Vault Container, run the following command:

vault policy write custom-mobile-apps-write-policy \
  vault/policies/custom-mobile-apps-write-policy.hcl

Now that we have a policy that allows writing into Vault, that policy has to be associated with the corresponding users. Vault does not come enabled with an auth backend for user authorization, so we have to enable one. For the purpose of this article, let’s enable the simple username/password authorization so we can create a test user account. From within the Vault Container, run the following:

vault auth enable userpass

Next, create the user with the write-only policy by running the following in the Vault Container:

vault write auth/userpass/users/myuser \
  password=mypassword \
  policies=custom-mobile-apps-write-policy

That’s enough to allow that user to write custom-mobile-apps secrets.

Secure Keychain Data

We now allow select users to write secrets into Vault. Remember, we want to write the customer’s encrypted keychain password. For the convenience of keeping the two pieces of data together, we also want to store the keychain alongside it. Both of these pieces of data are binary, which Vault cannot store. Instead, we have to base 64 encode it.

Here’s a script that does just that. I added a bit more code to pretend that I received the encrypted keychain password.

Let’s login as myuser in order to get a new client_token.

curl --request POST --data '{"password": "mypassword"}' \  
  $VAULT_ADDR/v1/auth/userpass/login/myuser
# copy the value for 'client_token' in the 'auth'
# object in the returned value
export CLIENT_TOKEN=<copied client token>

Now that we have the script, and we have a token that allows us to write to the keychains and APNS secrets, let’s do that:

# we want to pass in the client_token
# as the VAULT_TOKEN for the script to use
VAULT_TOKEN=$CLIENT_TOKEN bash ./secure-keychain.sh \  
  $HOME/myuser.keychain-db \
  $HOME/apns.txt

It will ask for the keychain password (hidden) and the company name associated with these signing credentials.

Now, whenever a customer updates the code signing assets, I only have to log into Vault and run this script.

Secure Code-Signing

We now have the keychain and its encrypted password in Vault. The build system needs to retrieve the keychain, unlock it, and then code sign the mobile application.

We use Jenkins for our build system. Jenkins builds can accept parameters, so we will create one to pass in a temporary token that can only be used to read the customer’s secret on a separate build machine that is locked down to only allow Jenkins access. The environment variables that Jenkins has to set are: VAULT_TOKEN and VAULT_ADDR.

I won’t show how to configure Jenkins here but instead assume that you can do something equivalent with your CI system. Instead, I will demonstrate how fastlane can fetch the customer’s credentials onto the build machine, decrypt the keychain password, and unlock the keychain in preparation for code signing.

If you’re following along, I suggest that you install and initialize fastlane. Respond ‘y’ to the prompt to “manually setup a fastlane config”. This will create a basic installation of fastlane with a basic Fastfile in the fastlane directory.

We begin by creating a new fastlane action named get_keychain_from_vault. A fastlane action is a Ruby file that is loaded into fastlane and encapsulates a single piece of functionality for maintainability and reusability. In the Terminal, change the working directory to the parent directory of your fastlane directory, and type:

fastlane new_action
# When prompted ‘name of your action:`, type get_keychain_from_vault

Here is the Ruby code for that fastlane action:

Vault Token Role

Finally, to create a token that can only be used by the CI build machines to read secrets, we have to create a token role.

First, we have to define what the token can have access to (remember, we only want it to be able to read the customer’s credentials). Here is the definition:

Then, from within the Vault container, write the policy into the system:

vault policy write custom-mobile-apps-read-policy \
  vault/policies/custom-mobile-apps-read-policy.hcl

After creating the policy that allows reading the customer’s secret, we want it to be associated with some entity. That’s where the Vault Token Role comes in — it allows someone to take on the role which can create a read token.

Let’s create a new token role with the read-only policy. From within the Vault container, run:

# create the token role and declare which policies it can access
vault write \
  auth/token/roles/custom-mobile-apps-read-policy-create-role \
  allowed_policies=custom-mobile-apps-read-policy

We now have a token role that can read the secret, so we have to give the users the ability to create a token that allows others to take on that role. That means a new policy has to be created:

Next, let’s create the actual policy that allows the creation of the token role. From within the Vault Container, run:

vault policy write custom-mobile-apps-read-policy-create-role \
  vault/policies/custom-mobile-apps-read-policy-create-token-policy.hcl

Finally, we have to update the user account to also permit the creation of the token roles. Still within the Vault Container, run:

# update the user with the policies
vault write auth/userpass/users/myuser \
  policies=”custom-mobile-apps-write-policy,custom-mobile-apps-read-policy-create-role”

The user can now create a token that can be used by the CI build machines to read the customer’s secrets. Since we just changed the user’s policy, that user has to re-login to get an updated token that is associated with both policies.

Once a new token is retrieved, run the following to get the token that the CI build system needs:

curl --header "X-Vault-Token: $CLIENT_TOKEN" --request POST \
  $VAULT_ADDR/v1/auth/token/create/custom-mobile-apps-read-policy-create-role

Once you get that token, put it into your CI system’s VAULT_TOKEN parameter so that it can be used by the build system. Here is an example Fastfile that fastlane will use to retrieve the keychain data to unlock the keychain safely and securely:

Let’s test this lane, which is like a command, to test the retrieval of a keychain, the decryption of the keychain password, and unlocking the keychain.

VAULT_TOKEN=$ROLE_TOKEN fastlane keychain_from_vault_test

Final Thoughts

I’ve given you a walk through of how Vault’s Token Roles can allow one user to grant another user (even a machine) access to secrets on a temporary basis. You can review all of the example code in this GitHub repo.

With Vault, there are a lot more options that you should consider. As demonstrated above, tokens can be restricted to specific CIDRs, but you can do a lot more. For example, you can make them valid for only 1 use, valid for a short period of time, etc. Take a look at what you can do with a policy associated with the Token Role to get a better idea.

Keep in mind that this is not the final result. In another article, I will demonstrate how we ended up using Vault’s AppRoles to improve the security around these secrets even further.