How to Store and Retrieve your Secret Keys using AWS Secrets Manager?

Published in

IntelliconnectQ Engineering

6 min readMay 29, 2021

The above diagram displays you can store credentials for a database in Secrets Manager, and then use those credentials in an application to access the database.

Let’s first see what’s AWS Secrets Manager.

Helps to protect secrets needed to access your applications, services, and IT resources.
Enables to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle.
Users and applications retrieve secrets with a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text.
In addition, Secrets Manager enables to control access to secrets using fine-grained permissions.

Benefits :

Rotate Secrets Safely
Manage Access with fine-grained Policies
Secure and Audit Secrets centrally
Pay as you go

Steps :

The database administrator creates a set of credentials on the Personnel database for use by an application called MyCustomApp. The administrator also configures those credentials with the permissions required for the application to access the Personnel database.
The database administrator stores the credentials as a secret in Secrets Manager named MyCustomAppCreds. Then, Secrets Manager encrypts and stores the credentials within the secret as the protected secret text.
When MyCustomApp accesses the database, the application queries Secrets Manager for the secret named MyCustomAppCreds.
Secrets Manager retrieves the secret, decrypts the protected secret text, and returns the secret to the client app over a secured (HTTPS with TLS) channel.
The client application parses the credentials, connection string, and any other required information from the response and then uses the information to access the database server.

Features of Secret Manager :

Programmatically retrieve encrypted secret values at runtime
Storing different types of secrets
Encrypting your secret data
Automatically rotating your secrets
Database with fully configured and ready to use rotation support
Control access to secrets

Accessing Secret Manager :

You can work with Secrets Manager in any of the following ways :

AWS Management Console
AWS Command Line Tools
AWS SDKs
Secrets Manager HTTPS Query API

Creating and retrieving a secret :

You can sign in to AWS as an IAM user with permissions to create and retrieve secrets in the AWS Secrets Manager console, or use equivalent commands in the AWS CLI.

Creating and storing your secret from the console :

Step 1: Configuring a test RDS database: Refer here

Step 2: Create Your Secret :

Open the Secrets Manager console
Ensure you set your console to the same region as you created the Amazon RDS database.
Choose Store a new secret.
On the Store, a new secret page, in the Select secret type section, choose Credentials for RDS database.
For the User name, Add Master username
For Password, type the same password that you provided for the master user when you created your database.
For Select the encryption key, leave it set to DefaultEncryptionKey. AWS bills your account if you use a custom master key (CMK) instead of the default CMK.
For Select which RDS database this secret will access, and choose the instance. Choose Next.
In the Secret name and description section, for Secret name, type MyTestDatabaseMasterSecret. Choose Next.
In the Configure automatic rotation section, disable rotation for now. Choose Next.
In the Review section, verify your details, and then choose Store.
Secrets Manager returns to the list of secrets, which now includes your new secret.

Step 3: Validate your initial secret

Before you configure your secret to rotate automatically, you should verify you have the correct information in your secret and can connect to the database.

To test your database connection, you can use any GUI-based application like, MySQL Workbench or DBeaver.

You can retrieve the secret by using either the AWS CLI or the Secrets Manager console. Then cut and paste the user name and password into the GUI-based application to test the connection.

Step 4: Configure Rotation for Your Secret :

To configure secret rotation

In the Secrets Manager console, choose your secret.
On the Secret details page, in the Rotation configuration section, choose Edit rotation.
On the Edit rotation configuration page, choose Enable automatic rotation.
For Select rotation interval, choose 30 days.
Under Select the secret will be used to perform the rotation, choose Use this secret.
Choose Save. Secrets Manager begins to configure rotation for your secret, including creating the Lambda rotation function and attaching a role enabling Secrets Manager to invoke the function.

Stay on the console page with the Rotation is being configured message until the message changes to Your secret <SECRET-NAME> has been successfully stored and secret rotation is enabled.

Retrieving your secret from AWS Secrets Manager

If not already logged into the console, go to the console and log into the Secrets Manager service.
On the Secrets list page, choose the name of the new secret you created.
Secrets Manager displays the Secrets details page for your secret.
In the Secret value section, choose to Retrieve secret value.
You can view your secret as either key-value pairs or as a JSON text structure.

Secrets Manager enables you to programmatically and securely retrieve your secrets in your custom applications.

Minimum permissions :

To retrieve a secret in the console, you must have these permissions:

secretsmanager:ListSecrets — Use to navigate to the secret to retrieve.
secretsmanager:DescribeSecret — Use to retrieve the non-encrypted parts of the secret.
secretsmanager:GetSecretValue — Use to retrieve the encrypted part of the secret.
kms:Decrypt — Required only if you used a custom AWS KMS customer master key (CMK) to encrypt your secret

To set these permissions,

Open the Secrets Manager console.
From the list of secrets in your account, choose the name of the secret to view.
The Secret details page appears. The page displays all of the chosen secret configuration details except for the encrypted secret text.
In the section, Resource Permissions :

Add following sample Permissions, you can edit permissions based on your requirements :

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Principal":{
            "AWS":"arn:aws:iam::123456789012:user/user123" => ARN of Account whose access keys will be used
         },
         "Action":"secretsmanager:GetSecretValue",
         "Resource":"arn:aws:secretsmanager:<your-region>:123456789012:secret:<secret-path>" => ARN of Secret
      }
   ]
}

To use these secrets in the application we can use AWS SDK and Secrets Manager’s API to retrieve the secrets, as given below :

// Use this code snippet in your app.
// If you need more information about configurations or implementing the sample code, visit the AWS docs:
// https://aws.amazon.com/developers/getting-started/nodejs/
// Load the AWS SDK
var AWS = require('aws-sdk'),
    region = "ap-southeast-1",
    secretName = "prod/payroll/vgerp",
    secret,
    decodedBinarySecret;// Create a Secrets Manager client
var client = new AWS.SecretsManager({
    region: region,
    secretAccessKey: secretAccessKey,
    accessKeyId: accessKeyId
});// In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
// See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
// We rethrow the exception by default.client.getSecretValue({SecretId: secretName}, function(err, data) {
    if (err) {
        if (err.code === 'DecryptionFailureException')
            // Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw err;
        else if (err.code === 'InternalServiceErrorException')
            // An error occurred on the server side.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw err;
        else if (err.code === 'InvalidParameterException')
            // You provided an invalid value for a parameter.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw err;
        else if (err.code === 'InvalidRequestException')
            // You provided a parameter value that is not valid for the current state of the resource.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw err;
        else if (err.code === 'ResourceNotFoundException')
            // We can't find the resource that you asked for.
            // Deal with the exception here, and/or rethrow at your discretion.
            throw err;
    }
    else {
        // Decrypts secret using the associated KMS CMK.
        // Depending on whether the secret is a string or binary, one of these fields will be populated.
        if ('SecretString' in data) {
            secret = data.SecretString;
        } else {
            let buff = new Buffer(data.SecretBinary, 'base64');
            decodedBinarySecret = buff.toString('ascii');
        }
    }
    
    // Your code goes here. 
});

So, we can retrieve and use our AWS Secrets as described above.