Securing your workflow using Vault Agent with GCP Auth Method on HashiCorp Vault
How HashiCorp solves the secure introduction challenge using the platform integration model.
Introduction
How does an application or machine first prove their legitimacy in order to acquire an access token to pull secrets? This is the premise for the secure introduction problem. There are two solutions to this problem: platform integration or trusted orchestrator. In this article, I will explore how we can use Google Cloud Platform (GCP) as the trusted underlying platform for a Vault Agent running on the client host.
Building on the Vault AppRoles example in my previous article, the client host running Jenkins needs to be authenticated to Vault first and have the correct policies to generate a SecretID before it can authenticate against the AppRole auth method.
Under the Hood
The Vault Agent automates the authentication process and manages token lifecycles, but it still requires a method to authenticate with. Because my example operates in GCP, I will be using the GCP auth method to authenticate. This solution to the secure introduction challenge is known as the Platform Integration model because Vault trusts the underlying platform upon which the compute engine is built. This is analogous to using a government-issued passport to enter or leave the country. There are 2 ways to authenticate:
- IAM service accounts
- Google Compute Engine (GCE) instances
The former authenticates against a specific service account. A JSON Web Token (JWT) for the service account is generated and sent to Vault which then verifies its validity with Cloud IAM.
The latter, which I will be focusing on, uses a JWT created and signed by Google for identifying instances, sends the JWT to Vault which is then used to verify against Google. The advantage over the IAM method of authenticating is that it must be done from a running GCE instance and hence there are additional security restrictions that can be applied such as GCE region, zones, labels and instance group.
Prerequisite
On the GCP side, you will need to enable the APIs:
- Identity and Access Management (IAM) API
And create a service account (i.e. vault-gcp-auth-verifier) in Cloud IAM with the following roles:
- roles/compute.viewer
- roles/iam.securityReviewer
You will also need the credentials file in JSON format for said service account.
Setup
Enable the GCP Auth Method:
vault auth enable -path=my-project-123 gcpHere, I am using my GCP project ID, my-project-123 as the custom path for organizing my GCP auth method endpoints.
Provide the credentials to allow Vault to acquire the JWT from the compute engine instance and view its metadata such as project ID, location, labels, etc.
vault write auth/my-project-123/config credentials=@credentials.jsonConfigure a role, specifying the gce auth type and any additional security restrictions you would like:
vault write auth/my-project-123/role/gce-role \
type="gce" \
token_policies="jenkins-agent" \
token_ttl=1m \
token_type=batch \
bound_projects="my-project-123" \
bound_labels="vault-auth:true" \
bound_regions="northamerica-northeast1, northamerica-northeast2"The options I have chosen to secure the GCP auth method is to limit the project, specific GCE label and regions that can authenticate against GCP. The Vault Agent will manage the token’s lifecycle. I also chose to use a batch token type along with a short token TTL. For a full list of configurable parameters, consult the API documentation.
Because the agent is to support the Jenkins AppRole, I have defined the jenkins-agent policy (where medium is my custom AppRole path) as:
path "auth/medium/role/jenkins/role-id" {
capabilities = [ "read" ]
}path "auth/medium/role/jenkins/secret-id" {
capabilities = [ "update" ]
}
Here is my vault-agent.hcl config:
I directed the agent to the Vault server and configured Auto-Auth using the GCP auth method and authenticating against the role I defined earlier. The resulting token will be written to a sink file and automatically rotated by the agent based on the token’s TTL. The listener stanza defines the IP address and port the agent will listen for requests (this is typically set to localhost with a non-standard Vault port).
TIP: access to the generated token file will be restricted to the vault user and group only by default. In order to provide the jenkins user access to the file, permissions will have to be relaxed (mode = 0644) and delegated to the jenkins group at the folder level instead. (i.e. chown vault:jenkins /path/to/dir and chmod 750 /path/to/dir)
How to Use
Running the Vault service in agent mode will allow it to connect, authenticate and manage the lifecycle of the token that gets written to the token file.
Set VAULT_ADDR=http://127.0.0.1:8100; the Vault Agent acts as your proxy for interacting with the Vault server, so requests should be made to the agent instead of the Vault server directly as you might otherwise would. Using the token file’s credentials will allow Jenkins to generate a SecretID and authenticate against the AppRole to be authorized to access the secrets it needs for the pipeline to progress.
Here is an example Bash script that executes all the aforementioned steps:
With the final token, you can now access the secrets it is authorized to retrieve.
Putting it All Together
Vault Agent and AppRole goes together like peanut butter and jelly; together they allow your machine to authenticate to Vault through the underlying cloud provider and enabling your application to generate secure, one-time passwords (OTP) to retrieve secrets with.
Each gatekeeper has its own set of criteria for successful authentication, providing defense in depth. Combined with a constantly rotated auto-auth token, OTPs, short-lived access tokens and perhaps even dynamic secrets — all happening programmatically — and you have a secure workflow that is also convenient.
Next Steps
HashiCorp has plenty of good documentation, tutorials and YouTube videos describing various Vault features with demos, so I encourage you to check them out.
