How to Manage Application Secrets with Hashicorp’s Vault.

In this article, I want to show you how to manage application secrets with Hashicorp Vault. As a software developer, especially a backend developer, there are so many values that your application uses, some of which you may not want the general public to see. These secrets, could include:

  • Database usernames, passwords, ports as well as the database name and the server’s address, where the database is hosted.
  • Secret keys for encryption and decryption.
  • Default admin passwords and emails
  • Server addresses, where an application is deployed as well as the usernames and passwords for logging in.
  • And many more…

Keeping track of these secrets become difficult, especially if you deal with numerous applications or work in large teams where different developers need access to these secrets with different levels of authorization i.e a database administrator vs the tech lead vs a junior developer. This is where Hashicorp’s vault comes in.

Introduction

To manage these secrets, I currently store them in a .env file both in development and production, something that I quite don’t like. I bet you do too. And then I have to make them simple, to avoid mistyping them into the .env file. But with vault, i can overcome both. So what is Vault?

Vault is an open-source tool used for securely storing and managing secrets.

What is a secret? Secrets, in the context of this tutorial, are securely-sensitive or personally identifiable info like database credentials, SSH keys, usernames and passwords, AWS IAM credentials, API tokens, Social Security Numbers, credit card numbers, just to name a few.

There are numerous components that make up Vault. In this tutorial, we will concern ourselves with only two concepts:

  • Storage; this describes where the secrets are stored by the Vault. It could be a file system backed ona our computter, Consul offered by Hashicorp (needs to be configured), a database such as PostreSQL or even AWS S3. In this tutorial, we will use the filesystem backend to store our secrets.
  • Secrets: this describes the type of secrets that are to be stored. This could be static (do not change over time) or dynamic (generated on the fly). These secrets vary from AWS credentials, database credentials as well as key value pairs. In this tutorial, we will only look at key value pairs.

Application Secrets

In this article, we will assume that a Flask application is to be deployed with a postgres database. If you are familiar with flask, then you know that we would need to give out at least:

  • The flask environment i.e development as FLASK_ENV
  • The flask app, as FLASK_APP
  • The database credentials, as POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_HOST, POSTGRES_PORT and the POSTGRES_DB.
  • The app secret as SECRET_KEY.

These values could be provided by exporting them before running the application:

export FLASK_APP=api/__init__.py

or using a .env file. In this tutorial, we will store the secrets using Vault and then when needed, read them from vault and then populate the .env file with the values.

Vault Configuration

To configure a vault server instance, we will use docker compose to spin up a vault server, then store our secrets in the created server. To get started at the comandline:

mkdir vault-consul-docker && cd vault-consul-datamkdir vault && cd vaultmkdir config data logs policies # you should have the folder structure below within the vault-consul-docker folder└── vault
├── config
├── data
├── logs
└── policies
  • The config folder will hold the vault server configuration, in this case, where to store the data as well as the logs.
  • The data folder will contain the data that we add to vault
  • The logs folder as the name implies will contain the logs
  • The policies folder will contain our policies. Remember when I hinted to limiting access to secrets by different team members in a project? Policies help with that

Let us create the configuration file that vault will use. Within the config folder, create a file called vault-config.json:

  • Here we use a file system as our back-end and the data is stored within the vault/data directory.
  • The server listens for connections on port 8200

Next, we create the Dockerfile that creates the vault server container. In the vault directory, create a file called Dockerfile and add the following:

  • We use an alpine base image
  • The vault version is then set, then the vault directory created.
  • We then add bash, that we will use later and wget for downloading the vault instance.
  • We then download and install vault, set the path to the vault directory and then copy the config file.
  • The vault port 8200 is then exposed for access to the container.

Let us then create the docker compose file for bringing up the container. Create a file called docker-compose.yml at the project root (the vault-consul-docker folder) and add the following:

  • We build the vault service using the above Dockerfile.
  • We then map port 8200 on our local computer to port 8200 of the container that is built.
  • We then map the vault volumes in the container to the directories on our computer.
  • We then set the command to execute when the container starts up.

To start the container:

sudo docker-compose up --build

If you followed the instructions as given out, you should now have a vault server running on port 8200.

Vault setup

Now that we have the vault server up and running, we need to set it up before we can store and access secrets. This involves:

  • Initializing vault
  • Unsealing vault
  • Logging in

First, let us initialize vault. This will give us a root token(for logging in) and 5 keys for unsealing vault. Let us access the vault command-line; in a different command-line tab:

sudo docker-compose exec vault bash # get a bash session on the running container

Then within the bash session, initialize vault:

vault operator init #initialize vault

You should get:

Keep those 5 unseal keys secret at a safe location. Without them you cannot unseal vault, and if vault is unsealed you cannot read its secrets or write to it.

Next, let us unseal vault. We will need three keys. On the bash session:

vault operator unseal

Then paste one of the unseal keys at the prompt. Due this three times with a different key each time. Once done, the vault should be initialized and unsealed:

Then let us log in:

vault login

Use the root token that you obtained earlier on.

Create and Read a secret at the Vault Commandline

With us logged in, it’s time to create and read a secret. We will create a key value pair, so let us us the kv engine offered by vault. We start by enabling it. Remember this is a static secret (does not expire). At the bash session:

vault secrets enable kv

Then let us create a key value pair, in this case, the flask environment:

vault kv put kv/vault-secret-management/local/flask_environment FLASK_ENV=development

In this case we create a new key value pair at the path kv/vault-secret-management/local/flask_environment . The path is composed of:

  • vault-secret-management, which is the current project name.
  • local, which tells me that this value is used during development on my local computer
  • flask_environment which is what the secret is keeping

I use this format for storing secrets since I create tons of flask applications, and host them on AWS, Linode and Heroku. Knowing what a secret is and which application and what environment it is for becomes easier with that format.

To read it:

vault kv get kv/vault-secret-management/local/flask_environment

You get back:

Well, that is how to set and read secrets using the kv engine. You can also set and read secrets using the vault UI or via HTTPS.

Read Secrets over HTTPS

I will not go through creating a secret using the UI or HTTPS. We will only read the secrets over HTTPS, since that is what we are concerned with. At a new command-line tab, use curl to access the vault server and read the secret, but first, you need to provide the root access token:

export VAULT_TOKEN=s.GMRc7gPtdE0rBNbmyt1AKecn # the root access token

Then read the secret:

curl \
-H “X-Vault-Token: $VAULT_TOKEN” \
-X GET \
http://127.0.0.1:8200/v1/kv/vault-secret-management/local/flask_environment

You get back:

And with that, we have set up and then read a secret using vault. Now try to set up and read the other secrets that we mentioned earlier on. Here is an example for FLASK_APP:

vault kv put kv/vault-secret-management/local/flask_app FLASK_APP=api/__init.py

And to read it:

curl \
-H “X-Vault-Token: $VAULT_TOKEN” \
-X GET \
http://127.0.0.1:8200/v1/kv/vault-secret-management/local/flask_app

Automate the reading of secrets

Finally, let us automate the reading of these secrets. Remember that at the start, we wanted our flask applications to read these values from a dotenv file using the dotenv python library. Create a file called run.sh in the project root and add the following:

  • When executed, the script first deletes the existing .env file
  • It then makes a call to the vault server to get the particular secret. The vault server returns a JSON response, which is passed using the jq utility. The jq utility is a command-line tool that is used to parse JSON. Once the JSON is parsed and the desired key obtained, this is written to the .env file.

To run the script, first install the jq utility:

sudo apt install jq -y # install jqchmod +x run.sh # make the script executable./run.sh

A file called .env is created in the project root. Anytime you use this .env file make sure that you add it to the .gitignore file. You get a file called .env with the following values (if you do not add the rest of the secrets like me):

This can then be picked up by your application.

With vault, you can now set secrets that are long and complicated and reuse them as much as you wan. You can even version them and read the versions that you want as well as set policies that govern access to these secrets.

Once done, exit the bash session:

exit

Then stop the running container.

To read them, you need to restart the container, then unseal vault.

Conclusion

In this tutorial:

  • We briefly went over secrets, and the current not so secure or scalable way of managing them
  • We then briefly went over Hashicorp’s vault
  • We then set up a very simple vault and used the key value engine to set up key value pairs
  • We read the secrets from the command-line then over HTTPS
  • We finally created a script that automatically loads the secrets into a .env file for our application.

In the next tutorial, we will deploy vault to a server on AWS or Linode and access it from a GitHub workflow and from a deployed application. We will also use a docker swarm with a consul back-end. To see the vault server secret management in use check out my tutorial series on deploying a production grade flask application to AWS.

I hope you enjoyed it and learnt something. Give it a clap or share it out and do not hesitate to reach out to me in-case of an issue. The code for this application is here vault-secret-management. I am Lyle, a junior software engineer with a passion for developing, testing and deploying scalable services. You can find me on twitter, linkedin, github and here’s my portfolio. See you next time.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Lyle Okoth

Lyle Okoth

A creative software engineer that loves solving problems involving algorithmic design and cloud computing.