Vault: Securely manage sensitive data

Pranay Sankpal
Nov 20, 2018 · 5 min read

Overview

In this article, we are going to explore HashiCorp’s Vault and understand what problems it solves.

  • The traditional approach of storing an application’s secret.
  • What is Vault and what problems it solves?
  • Setting up the Vault.
  • Interacting with vault.

Disclaimer: This article is not aimed at providing Vault tutorial. We suggest going through the official documentation for the tutorials.

The Traditional Approach and Problems with it

An application depends on many resources to function properly such as databases, third party API’s, logging and monitoring tools, and many more. When the application enters into running state, it uses secrets to interact with these required resources.

There are many ways of storing this information. Most commonly this information is stored in application’s config files. It is the easiest way to store the secret data and many frameworks support this approach. For each application environment, such as Dev, QA and Prod have to maintain it’s own secrets for the application. It is easier to come across a situation where different config files with secrets for each environment are present in the code base. If someone has access to the application codebase, he/she has access to all the application’s secret data such as database password, API’s auth headers, credentials to tools used by the application.

Imagine a situation where someone gets access to the production database and manipulate the production data or tries to export user or customer details or tries to extract contact details. Things can drastically go wrong and it can not only damage the company’s reputation but also the entire ecosystem.

Another most commonly used approach to store application secrets is using .env file. With this approach, the secret data related to the particular environment is stored in the .env file and these env files are not kept in the same repository where the application code is stored. In this way, a person having access to the particular environment will be able to access secret data of that environment. This avoids the situation of getting access to the Prod environment secrets to someone using the QA or Dev environment.

Though storing secrets in .env file is better approach compared to storing it in an application, these .env files still need to be pushed to some repository in version control system. Since it is difficult to enforce ACL, if someone gets access to the repository then he can access the information.

To tackle this situation, files with secret data are encrypted. This approach doesn’t make any difference as the application requires access to the master key to access the secrets. The entire approach creates a false sense of security.

What is Vault and what problems does it solve?

Hashicorp’s Vault is a tool designed to store and access application’s sensitive data securely. It helps us to build a centralised secret management service. Vault can generate the secrets, encrypt the data, limit the access to the stored data and also it can help in revoking the access.

It is very critical for any business to know who is accessing the secrets and it becomes difficult to implement with the approaches mentioned above. Vault solves this problem by collecting and publishing audit logs. Audit logs contain request and response object of every interaction with the Vault. By default, sensitive information is first hashed before logging in the audit logs.

Maintaining versions of secrets and when needed roll back to a particular version of the secret data is very important in the production environment. It is necessary to recover from unintentional data loss and having a correct version of the secret available in the production environment. It is possible for more than one user to write the same secret at the same time which overrides the data. Vault allows retaining a configurable number of secret versions. This enables older versions’ data to be retrievable in case of unwanted deletion or updates of the data. In addition, its Check-and-Set operations can be used to protect the data from being overwritten unintentionally.

Along with giving access to a resource, it is also important to revoke the access. Many times developer needs access to storage platforms like s3 or some features in AWS like SQS or Lambda for an experiment. It is necessary for the growth of the organization but at the same time, it is also important to revoke access to these features once experiments are conducted. Vault makes it very easy by giving a lease to some secret types. The user is able to consume data for the valid time duration or Time-To-Live (TTL) specified in the lease and once it is expired, access is revoked automatically. vault revoke command can be used to manually revoke the access.

Setting Up The Vault

For setting up the vault, follow the standard document.

For dev: https://www.vaultproject.io/docs/install/index.html

For production: https://github.com/hashicorp/terraform-aws-vault

Interacting with Vault

Login with the root token. This is the only time you will use the root token. Using root token is not a good practice and this should be avoided as much as possible.

Create the following policy and save it as secret_access_policy.hcl

path “secret/*” {
capabilities = [“create”, “read”, “update”, “delete”, “list”]
}

Once the policy is created, add this policy to the vault.

> vault policy write secret_access_policy secret_access_policy.hclSuccess! Uploaded policy: secret_access_policy

Now create a user, to do this, we first need to enable userpass auth method.

vault auth enable userpass

Now run following command to create a user.

> vault write auth/userpass/users/pranay \
password=mypwd \
policies=secret_access_policy
Success! Data written to: auth/userpass/users/pranay

Once the above steps are done, log in using the created user.

> vault login -method=userpass \
username=pranay_test \
password=mypwd
Success! You are now authenticated.

Now you are ready to create secrets and access them. It is important to decide the structure of the secret paths to store secrets. For us, secret/${ENVIRONMENT}/${APP}/key vaule=${VALUE}worked.

> vault kv put secret/PROD/MyMusicApp api_key=abc api_secret=pqrSuccess! Data written to: secret/PROD/MyMusicApp

Read stored secrets

> vault kv get secret/PROD/MyMusicApp======= Data =======
Key Value
--- -----
api_key abc
api_secret pqr

Now, delete the created secrets

> vault delete secret/PROD/MyMusicAppSuccess! Data deleted (if it existed) at: secret/PROD/MyMusicApp

For tutorials, please refer official document.

Conclusion

We found vault is very useful when it comes to managing application’s environment variables and sensitive data.

Join our community Slack and read our weekly Faun topics ⬇

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇

Faun

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts

Pranay Sankpal

Written by

Faun

Faun

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade