Essential Patterns of Vault — Part 1

Jake Lundberg
Jun 24 · 7 min read

Learning HashiCorp Vault can be a daunting task when approached for the first time. While Vault can seem quite difficult on the outside, there are some distinct and repeatable patterns that are simple to learn and which can greatly accelerate adoption. This is a two part blog post. Part 1 will focus on the high level explanation of the Essential Patterns. Part 2 will take these patterns and apply them with code examples and detailed explanations.

When describing Vault to my customers, I usually start the conversation with the Vault Triangle. This triangle contains a snapshot of Vault’s current set of operations in one place and is a great visual representation for the typical flow of information. While Secret Engines and Authentication Methods will increase over time, the discipline for using them will remain fairly similar.

In its simplest form, the process of using Vault is:

  1. Be familiar with the documentation
  2. Enable and configure Secret Engines
  3. Enable and configure Authentication Methods
  4. Define and create policies
  5. Authorize by tying policies for accessing secret engines to authentication methods.
  6. Do positive and negative tests to prove your configurations allow access to the needed secrets as well as disallow access to unneeded secrets.

In a nutshell, that’s it.

Docs and Learn

This shouldn’t be a surprise to anyone, but the first discipline of Vault (or any other tool for that matter) is knowing where to find information on your own. While community information like HashiCorp Discuss, StackExchange or Reddit, as well as Slack channels like Hangops or SweetOps, are great for asking people online, our first stop should always be the Vault Documentation and Vault Learn.

Vault Docs are as you’d expect, the documentation for the product. Inside each section, you will find an introduction to each subject as well as very detailed API documentation (usually linked at the bottom of the intro section). Usually when I’m working on a particular solution, I’ll have both open for giving examples as well as digging deep into any special parameters or outputs given from the API.

Learn has a fantastic amount of curated material to walk any new (or old) users of Vault through all subjects it covers.

Now that you’re familiar with the docs, let’s talk about the reason everyone uses Vault: secrets.

Secrets

First and foremost, Vault is a secrets management solution. One of the main objectives is to reduce secrets sprawl. Defining this is outside the scope of this blog, but I did a blog with Microsoft on the subject: How to identify and eliminate secrets sprawl on Azure with HashiCorp Vault — Open Source blog.

So naturally, the first task we need to do is define what kinds of secrets we want to manage and then enable them.

Vault allows you to manage:

  • Key Values (simple or complex)
  • Dynamic database/datastore credentials
  • Dynamic cloud credentials
  • PKI Certificates
  • SSH CA Certificates
  • Encryption as a Service
  • Consul and Nomad ACL tokens
  • Custom Plugins

Most people will start off with Key Values first, but the rest of the engines are widely used as well. Dynamic secrets are amazing because they allow you to use Vault as a Secret Broker to other technologies that normally have static secrets. Vault will both create and remove credentials based on any number of rules you set. Time and usage count-based bounds are very popular.

Once you’ve enabled the secrets for your organization, the next task is to figure out how you are going to allow humans and machines to access those secrets.

Authentication

Before you can access anything in Vault, you need a special access key called a Token. In order to get that token, you have to authenticate to Vault. You could distribute tokens directly, but creating, storing, and distributing them securely can be non-trivial. Authentication methods then, are convenience methods for retrieving tokens.

Vault has a wide array of Authentication Methods to include (but not limited to):

  • LDAP/AD
  • Okta
  • Kubernetes
  • AWS IAM
  • GCP IAM
  • Azure MSI
  • GitHub
  • Generic JWT and OIDC methods
  • PKI Certificates
  • Username and Password
  • AppRole

You can easily enable as many authentication methods as you need; Vault is very flexible.

Just authenticating with Vault is not enough however; you also need to let Vault know what the entity accessing it should be allowed to interact with. Think of it like the Token you are given is blank with no permission to do anything, it needs to get permissions on what it can access.

Authorization

The way you get the permissions for the Token is through Policies.

Everything you access in Vault is done at a path. Under the hood, Vault uses radix trees for its data storage. This allows us to quickly map policy definitions to secrets and administrative functions at scale and with many concurrent requests.

Policies are the way of defining which paths in Vault can be accessed. You can create as many policies as you want with as many path combinations as necessary.

Example Policy:

# KV V1 Examples
path “secret/db1/finance/*” {
capabilities = [“create”, “read”, “update”, “delete”, “list”]
}
path “secret/it/*” {
capabilities = [“read”, “list”]
}
path "secret/it/supersecret/*" {
capabilities = [ "deny" ]
}

Once created, the next step is to tie a policy or set of policies to the authentication methods enabled on the system. Generally speaking, most authentication methods have a concept of roles and those roles are what policies are tied to. API documentation is rich, so any deviation to this naming convention is easy to catch.

One extremely powerful concept is that policies can use Policy Templates. Templates allow us to accelerate policy distribution by using variables instead of hard-coded paths. This allows us to use common classification patterns, like LDAP groups or Kubernetes namespaces, to define access versus explicitly writing paths. Using templates does require some understanding of Vault Identity, but in our examples, I’ll show some ways to accelerate how to learn this arcane subject.

Administration

This section doesn’t describe administration functions, but more points out that admin endpoints are also just paths within Vault. To allow admin access is to just allow access to the proper paths. For example, all root protected paths that require sudo access can be found at Root Protected API Endpoints.

Tests

Once you have everything setup, it’s a good idea to test your setup to ensure authentication and authorization both give access to needed secrets as well as deny access to unneeded secrets. We’ll go more in depth on this in Part 2.

Putting it All Together

A basic instruction set for implementing a Vault solution might be:

  • Enable K/V and Dynamic Database Secret Engines
  • Enable LDAP Auth Method
  • Define and create policies to access specific paths within each secret engine
  • Map policies to authentication methods
  • Do positive and negative policy tests

Additional food for thought

While the information I’ve given so far encompasses the main patterns we need to understand to be successful with Vault, there are a few other things worth pointing out that I find key to using Vault over time:

Running Vault in Dev Mode

It’s very easy to experiment with Vault when it comes to learning just about anything about it. Dev mode is in memory, doesn’t require unsealing, and allows for configurable root tokens, which eliminates the need to create hacky workarounds for unsealing and parsing out root tokens when doing development. The beauty here is that you can quickly iterate on automating processes as you can start with a clean slate as often as you like. In fact, when creating automation, this is by far the best technique to ensure your automation is complete as you always start with a fresh slate and don’t accidentally leave any configurations behind.

Though this should be obvious: DO NOT RUN THIS IN PRODUCTION!

Vault Agent

Vault also has an agent which can be used to simplify client-side operations such as:

  • Authenticating with certain auto-auth functions like AWS, Azure, or Kubernetes
  • Maintaining lifecycle of Vault tokens
  • Agent will both introduce as well as renew tokens
  • Caching leased secrets like dynamic DB credentials and PKI certificates

Before Vault Agent, it was up to the user to come up with a solution to do the stuff above. You’re still welcome to write your own tools of course, but Vault Agent can save you some effort.

Consul Template and Envconsul

Though both of these tools currently still have Consul in the name, they also work quite well with Vault. It’s possible the names will change in the future.

Consul Template can be used to read both Consul K/V values as well as Vault secrets. These values can then be used to fill in templated files to be used for application configuration. This can help accelerate Vault adoption as any application that reads a config file can be Vault enabled with little effort. Consul template is aware of secret leases as well and can renew secrets automatically when run as a daemon.

Much like Consul Template, envconsul reads from both Consul and Vault and creates environment variables that can be used for injecting into applications at start time, or possibly being re-read during run time for applications that can do this. For people focused on creating twelve-factor apps, this could be the tool for you.

Don’t touch that dial!

Now that we’ve gone over the core patterns for Vault adoption, I’ll punt until the next blog post, The Essential Patterns of HashiCorp Vault — Part 2, to give a detailed example of the Putting It Together section above.

Lastly, let me know if you ever actually used a dial to change TV channels.

HashiCorp Solutions Engineering Blog

A Community Blog by the Solutions Engineers of HashiCorp and Invited Guests

Jake Lundberg

Written by

Human

HashiCorp Solutions Engineering Blog

A Community Blog by the Solutions Engineers of HashiCorp and Invited Guests

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade