Store and Rotate API Keys with AWS Secrets Manager
There are many options when it comes to managing API keys in AWS. It’s a spectrum of good and bad with trade-offs in performance, cost, security, and complexity.
At the bad end of the spectrum are simple solutions like hardcoded strings and configuration files committed to version control.
This post isn’t intended to be a comparison of solutions. Instead, I’m simply going to show you how to store, access, and automatically rotate API keys using AWS Secrets Manager, which was introduced in April 2018.
AWS Secrets Manager
Secrets Manager is relatively new, so you may not have heard of it before.
Obviously, it’s a secrets management service. It enables you to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle.
Using Secrets Manager, you can secure, audit, and manage secrets used to access resources in the AWS Cloud, on third-party services, and on-premises.
That’s all you need to know going into this. I’ll build up your knowledge along the way.
Using Secrets Manager
Let’s start by looking at how you’d use Secrets Manager from your code. The diagram below shows a Lambda function that calls Secrets Manager to get the API key it needs to access an external API.
The GetSecretValue command has a single required parameter named
SecretId. Secrets can be identified using their name or ARN.
When you create a secret, you give it a name on to which AWS appends a hyphen followed by six random characters (code in the ARN below).
Regardless of whether you use the secret’s name or ARN as
SecretId, you don’t have to include the hyphen and code (though AWS recommend you do).
Secrets can have a string or binary value. The response of
GetSecretValue will have either a
SecretBinary property, respectively.
The AWS console only supports string values containing plaintext or JSON formatted key-value pairs. Binary secrets must be managed via the API.
To understand secret rotation, you need to know what secrets are made of, so let’s take a look. I’ll go into more detail after a quick summary.
- ARN & Name: Unique secret identifiers.
- Description: A human-friendly description of the secret.
- KMSKeyId: (Optional) The ARN of the KMS key that Secrets Manager uses to protect the secret.
- Rotation Configuration: (Optional) How frequently the secret is automatically rotated and which Lambda function is used to perform the rotation.
- Timestamps: Last accessed, last changed, and last rotated timestamps.
- Tags: Key-value pairs just like many other AWS services.
- ID: Unique version identifier. Currently a UUID by default.
- Staging Labels: List of strings.
- Secret Value: A string or binary value.
Versions and Stages
A secret can have one or more versions, each of which contains a secret value. In fact, it’s the versions that contain the value and not the secret itself.
A secret must have at least one version, and one of its versions must have the
AWSCURRENT staging label. Therefore, when you create a new secret, Secrets Manager automatically creates a version and gives it the
Each version can have up to 20 labels, but only one version can have each label at a time. Another way to look at it is that each stage can only have one version at a time. It’s a one-to-one mapping.
GetSecretValue example at the top of this post, we could have additionally passed in either
VersionStage. Secrets Manager would have returned the value of the version with the given ID or stage label. Since we didn’t, the default version with the
AWSCURRENT label was returned.
Let’s walk through a simple example to store an OAuth access token and refresh token.
In the Secrets Manager console, click Store a new secret on the right.
Select Other type of secrets (e.g. API key). Then add your access token and refresh token. You’ll need to click Add row or use the Plaintext view. You can store up to 7168 bytes in each secret.
KMS encryption is out of scope here, so leave DefaultEncryptionKey selected. The default means Secrets Manager creates and manages a new encryption key for each account in each region.
Finally, click Next to move on.
You need to give the secret a unique name. It only needs to be unique to the account and region, not globally like S3 buckets.
It’s recommended that you choose a naming convention and stick to it. In the placeholder text, we see
prod/AppBeta/Mysql which is using a slash to create a hierarchy. This is great for simple namespacing, but it's better to use tags for anything more complex.
Tags are great for things like grouping secrets, cost allocation and tracking, or recording ownership. You can even use IAM policies to restrict access to only secrets with particular tags.
A good name, description, and tags can help developers discover existing secrets instead of creating new ones.
Clicking Next takes you to the last page where you can configure automatic rotation. You can’t configure automatic rotation yet since you haven’t created a Lambda function, but let’s look at the options.
With Enable automatic rotation selected, you can choose a rotation interval of 30, 60, or 90 days. Alternatively, you select Custom and enter a number between 1 and 365 days. You can’t have an interval of less than a day at the moment.
Next, we’ll go into detail on the Lambda function and how Secrets Manager invokes it during the rotation process.
The last page allows you to review the configuration and provides some code examples for accessing the secret’s value from your application.
If you configured automatic rotation, the secret will be rotated immediately upon clicking Store.
The Rotation Function
Secrets Manager decides when to rotate your secret based on the interval you configure. Rotation happens randomly during a 24 hour period. That is, if you select a 7-day interval, your secret will be rotated every 144–168 hours.
There are four steps in the rotation process. Secrets Manager will invoke your Lambda function sequentially to perform each one.
Let’s walk through a rotation of the
prod/foo secret below. This secret has a single version
aaaaaaaaaaaaaaaaaaaaaaaa with the label
Step 1: createSecret
When Secrets Manager decides its time to rotate your secret, it generates a new version ID, a UUID like
bdb9c291-afa2–4435–822b-6240dc732caf. To make things simpler, we’ll use
Secrets Manager then invokes your Lambda function with the following input.
createSecret and the newly generated ID is passed in as
SecretId is the ARN of the
On this step, your Lambda function needs to do two things:
Create a new secret value
The exact implementation will differ by API, but let’s say our access tokens are created by sending an HTTP POST to the
/tokens resource of the API. In our example, the HTTP request must contain a
You should store the
client_secret in a separate secret, sometimes called a master secret. You then include the ARN or name of the master secret inside your main secret (
This keeps the client credentials isolated and safe. It also allows your Lambda function to be reused to rotate other secrets.
Your function needs to:
- Get the current secret value using
- Fetch the master secret using
client_secretfrom the master secret.
- Call the API’s
- Extract the new
tokenfrom the HTTP response.
Store the new secret value
Now that you have the new access token, you use the
PutSecretValue command to create a new version of the secret.
This command takes the
ClientRequestToken from your function’s input.
SecretString will contain the new token and a copy of
master_secret_id (for the next rotation). Lastly, the
VersionStages list will contain one label,
You can see below that
ClientRequestToken became the new version’s ID.
Step 2: setSecret
When rotating an API key, you usually won’t need to do anything for this step.
The input to your function will look like this. Note that only
step has changed. That’s the case for all invocations.
setSecret step is used for rotating other kinds of secrets. For example, when rotating database credentials, you generate a new password in
createSecret, and create the database user with that password during
In our scenario, there is nothing to do here.
Step 3: testSecret
Again, only the
step property of the input will have changed:
This step is your opportunity to ensure the new secret works as expected. In our case, we should test that we can successfully call the API with the new token.
To ensure the new secret is 100% working, AWS recommends you perform every action your application will. Obviously, some actions are destructive, so just do what you can to be confident.
Your function needs to:
- Get the new/pending secret value using
ClientRequestToken. By passing
ClientRequestTokenyou are saying you want the value of the secret version you created in
- Call the API using
tokento ensure it works as expected.
Step 4: finishSecret
The input to this final step is like the others. Only
step has changed.
In this step, your function will move the
AWSCURRENT label to the new version using the
Until now, clients have been reading version
aaaaaaaaaaaaaaaaaaaaaaaa because it had the
AWSCURRENT label. Subsequent
GetSecretValue calls will now return
UpdateSecretVersionStage has four request properties:
VersionStage— The label you want to move.
AWSCURRENTin this case.
SecretId— The ARN or name of
prod/foo. Get this from the input.
MoveToVersionId— The ID of the version we want to move
AWSCURRENTto. In this case, it’s
ClientRequestTokenin the input).
RemoveFromVersionId— The ID of the version
AWSCURRENTis currently assigned to:
Note that you’re not given the ID of version
aaaaaaaaaaaaaaaaaaaaaaaa, so you’ll need to use the
DescribeSecret command to get it. That command takes a
SecretId and returns the metadata and version details described in the Secret Recipe section further up.
Specifically, the response will have a
VersionIdsToStages property. This is a mapping from version ID to a list of staging labels.
aaaaaaaaaaaaaaaaaaaaaaaa": [ "AWSCURRENT" ],
"bbbbbbbbbbbbbbbbbbbbbbbb": [ "AWSPENDING" ]
Following the call to
UpdateSecretVersionStage, our example secret will look like this:
aaaaaaaaaaaaaaaaaaaaaaaa has been automatically given the
AWSPREVIOUS label. This helps to identify the last good version in case you need to roll back.
The Next Rotation
Subsequent rotations are all the same, but you may be wondering what happens to the old versions.
After the next rotation, you’ll have three versions. Since there is a one-to-one mapping between versions and labels, only
bbbbbbbbbbbbbbbbbbbbbbbb can have
aaaaaaaaaaaaaaaaaaaaaaaa without a label.
aaaaaaaaaaaaaaaaaaaaaaaa will not appear in the
DescribeSecret response, but can still be read. However, Secrets Manager will eventually delete versions without labels.
There is a 30-day free trial period when you first start using Secrets Manager, so there’s no reason not to give it a go.
After that, you’ll pay $0.40 per secret per month. It’s important to note that this is per secret not per secret version, so rotating and storing previous versions will not cost extra.
Every 10,000 API calls will cost you $0.05. It’s recommended that you cache secrets where possible. AWS offers a Java and JDBC caching library which caches secrets for 1 hour, which is good guidance if you use another language.
When using Lambda, retrieve your secrets outside the handler method and reuse them on subsequent invocations.
Secrets Manager has a short but steep learning curve. In my opinion, it’s worth it for the robust automatic rotations, on-demand rotations, fine-grained security, and ability to store complex secrets as JSON.
I hope this post has helped you in some way. Please let me know if you still have any questions. I’ll happily help and update my post.