Using AWS Secret Manager to centrally manage development environment variables
As developers, we often face challenges in managing environment variables during the development phase. Keeping sensitive information secure and up-to-date becomes increasingly complex as the application grows. In this blog post, we delve into the step-by-step process of configuring AWS Secret Manager, which significantly improves the development process by securely storing and retrieving secrets.
Setup Guide
To configure this for your projects, follow the steps below.
1: Install and configure the AWS CLI
The AWS Command Line Interface (AWS CLI) is a unified tool to manage your AWS services. To install this, simply follow the setup guide.
2: Add your environment variables to AWS Secrets Manager
The next step is to add your environment variables to AWS Secrets Manager.
3: Add a list of secret names/keys to your project
Your project will need to be aware of what secrets need to be fetched at runtime. In the Retail team, we achieved this by creating a config similar to below.
export const SECRETS = {
API_KEY: "/secrets/api-key",
SECRET_TOKEN: "/secrets/secret-token",
};In this example, the keys API_KEY and SECRET_TOKEN will eventually map to process.env.API_KEY and process.env.SECRET_TOKEN. The values /secrets/api-key and /secrets/secret-token map to the secret names in AWS Secrets Manager.
4: Install the AWS packages
Run the following command to install the AWS SDK to your project
npm install @aws-sdk/client-secrets-manager5: Add a script to fetch the secrets
The next step is to create a script that uses the AWS SDK to fetch the secrets.
import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';
import * as Secrets from 'path/to/secrets/config';/** Create an AWS Secrets Manager client */
const client = new SecretsManagerClient();/**
Retrieves a secret value from AWS Secrets Manager using the specified key and SecretId.
@param {string} key - The key associated with the secret value.
@param {string} SecretId - The identifier of the secret.
@returns {Promise<string>} - The secret value retrieved from AWS Secrets Manager.
*/
async function getSecret(key: string, SecretId: string): string {
const command = new GetSecretValueCommand({ SecretId });
const { SecretString } = await client.send(command); return SecretString;
}/** Retrieves all secret values from AWS Secrets Manager using the config. */
async function getSecrets() => {
try {
const promises = Object.entries(SECRETS).map(([key, value]) => getSecret(key, value)); await Promise.all(promises);
} catch (error) {
// Handle error
throw error;
}
};/** Executes the function */
getSecrets();
6: Update your `package.json` to run the script when starting in dev mode
The last step is to update your start script to fetch the secrets before starting your app.
For example, if the npm script was previously:
"start": "ts-node-dev src/index.ts"It would then become:
"start": "AWS_PROFILE=profileName ts-node -r tsconfig-paths/register path/to/fetchSecrets.ts && ts-node-dev src/index.ts"This will then start pulling the secrets directly from AWS whenever you run `npm start`.
Considerations
AWS SDK Authentication
If you need to extend the code to include authentication via Cognito, SSO, and other providers, please check out the @aws-sdk/credential-providers package. The docs are available here.
Hot reloading
As running most applications in development mode supports hot reloading, you must ensure that you are not re-fetching the secrets for every server restart. The Retail team achieved this by ordering the fetchSecrets script before running the main app.
Session Expiration
When authenticating with aws-google-auth, the session credentials have a 6-hour validity period. This will mean that developers will need to show more often to be able to run apps. This is a slight barrier when compared .env files which do not require regular sign-in/authentication.
Information has been prepared for information purposes only and does not constitute advice.
