Vault Onboarding: Namespaces and More
When I get asked a question a couple of times, I tend to think that maybe we missed something on documentation. That being said, there are certain things that belong in an organisation’s run book but don’t have a natural place in the software vendor’s documentation; consequently, they generally end up in the methodology section. Over the last couple of years, HashiCorp has compiled much of that content that seeds an organisation’s run book into how to operate our solutions as central capabilities.
There is a question that always comes up as soon as organisations start using Vault: “How can we do it in a more efficient manner?” If you are doing it right, Vault has a number of operational procedures that you need to follow (Backup / Restore, DR, Upgrades, Key Rotation, etc.), which becomes quite burdensome when you have a large number of Vault clusters running in a single organisation. (I still think the worst thing I heard is: “We are running 15 Vault clusters and every single one of them is configured differently and maintained by a different team.”) The “centralise it” solution seems kind of obvious, but in some organisations, it just simply leads to frustration and ticket purgatory. From a consumer perspective, they go from “I’ll push this change into Vault policy” to “I’ll submit a ticket and put my deployment on hold for a week”.
So how are most Vault mature deployments handling this?
Very early on in Vault’s design, we realised that consumers would be way more likely to download and start using an open source solution like Vault themselves rather than wait for a centralized service to be available. Later, as we thought about how these consumers could transition to a centrally managed service, we realized they would want to retain the agility they had previously had while unloading the operational overhead to a centralized team. This led us to ship Vault Namespaces.
The core idea behind it is this: How can we partition Vault in a way where certain capabilities can be delegated to a consumer while retaining central management of the solution?
I’m not going to do an in-depth overview on best practices for running Vault (mostly because I’ve already documented most of them here).
So, assuming you have Vault up and running right now, let’s look at the people that are going to consume that service.
- You will have certain teams that have poor secret management hygiene. They potentially come from different backgrounds and are barely following your current secrets management practices (if any).
- There will be certain individuals that are managing secrets in other systems.
- You will have people that have different levels of awareness of Vault.
- Ultimately, you might have teams that are using Vault already and want to continue doing so.
Namespaces are a way to structure and consume Vault as a central capability, but they are by no means a silver bullet. For your teams using Vault already, most organisations choose to migrate them to their own namespaces under a certain set of guardrails. As an example, we are working with organisations that are giving full control of Namespaces to specific teams but are pre-approving the types of authentication methods and secret engines they can use as well as the kind of key material they can use with Vault’s Transit secrets engine. You can limit these both with Vault ACL Policies or Sentinel Eendpoint Governing Policies.
As an example, in the Transit secrets engine, you might want to only allow AES256 or strong RSA:
# Create key material, non deletable, non exportable in unencrypted fashion, only aes-256 or rsa-4096
path "/transit/keys/*" {
capabilities = ["create", "update"]
allowed_parameters = {
"allow_plaintext_backup" = ["false"]
"type" = ["aes256-gcm96", "rsa-4096"]
"convergent_encryption" = []
"derived" = []
}
}When we work with consumers who are less familiar with Vault, we generally find that they are wrapped up in a “common” workspace, in which specific functionality in the form of secret engines is made available and managed centrally. Their secrets are generally static secrets, though it is becoming more and more common that consumers leverage static roles to store their existing service accounts for databases and have Vault rotate them periodically. For this type of consumer, authentications methods are set up in advance and policies are templated in order to reduce administrative overhead and streamline the adoption of Vault as their secrets management solution.
It is very common for consumers to move between these categories. As they become familiarised with Vault and their comfort level increases, they may require more flexibility and could potentially be migrated into their own Vault namespaces. Most service owners are actually setting up an internal qualification programme, in which they can assess the proficientcy of consumers in managing their own Vault Namespaces. (We also provide training and workshops that can be used to do that assessment.)
Going back to one of my initial statements, Vault Namespaces are ultimately not a silver bullet, but it is the best way for your internal customers to consume Vault in a very agile manner. There will be some teams that will be extremely interested and ramp up their skills, and others that ultimately just want to store their secrets in a secure manner. Always remember your priority, which is getting your secrets under management, and whenever possible, try to reduce the operational overhead of doing so.
