Secrets as code with Mozilla SOPS and AWS KMS
How to securely manage your secrets inside your application workflow
A simple question that pops in every application development cycle is: where do I keep the credentials/secrets used by my application?
Before we dive into the “world of secrets” lets align some concepts first: in the context of this post, what is a secret?
We’re going to assume that a secret is a piece of information that will be used to access some resource (like databases, apis, etc.) and cannot be made public for security reasons.
In this way, when we say “Secrets as Code”, we’re talking about one of possible methods on how to keep and retrieve those secrets to be used by an application. Before we dive in the main topic, let’s see some ways on how to manage the secrets:
- By writing them directly on source code(Hard Coded): the worst possible option, it’s the poor’s man’s choice on how to do it and represents two main problems:
- Security itself, everyone that has repository access can see all credentials and can use them freely if they wish (in some cases people tend to rely on enforced access by IP or other methods to feel ‘safer’);
- Reusing or making the application easily “deployable” into more environments becomes a nightmare, because you need to edit source code every time.
2. By keeping it on a different place than the source code: In this approach your application expects that credentials are available on filesystem or as environment variables but they aren’t related to the application’s lifecycle; someone/something somewhere keeps the credentials updated.
3. By keeping it in the same repo of your source code (and read them from the application or inject as environment variables): now you’re saying that your secrets have a lifecycle that is almost transparent and you need to open a merge/pull request to make the changes in a gitflow based workflow.
In this way, you move into making your VCS as the source of truth for all your code and its secrets.
In this post, we’re going to see how to leverage Mozilla SOPS and achieve a state where you can commit and manage your secrets in conjunction with the source code.
What is Mozilla SOPS?
Mozilla SOPS is a cli tool to works with filetypes that relies on key:value format (json, yaml, env) and does that by **encrypting only the values**, allowing us to see the keys and thereby comprehend the set of secrets that are present on that file without leaking the values.
An simple but effective example on how to use can be viewed here (from sops github page):
What do we need?
To make things work we’re going to need at least:
- An AWS account with privileges to create an IAM User and a KMS Key using cloudformation templates;
- AWS Cli installed and configured (https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html);
- A key to encrypt the values: in our case, we’re going to use a AWS KMS because it is a simple way to have an certified key (FIPS 140–2), works well programatically and is supported by SOPS.
- The sops binary, which can be download from https://github.com/mozilla/sops/releases or through
brew install sopsif you’re using MacOS.
Creating the IAM User
An IAM user will be used to encrypt and decrypt the secrets. The first thing that we need is to create this user. You can create it by using the following cloudformation template:
Resources:
OperatorUser:
Type: 'AWS::IAM::User'
Properties:
UserName: 'kms.user' OperatorAccessKey:
Type: 'AWS::IAM::AccessKey'
Properties:
UserName:
!Ref OperatorUserOutputs:
OperatorAccessKeyOutput:
Value:
!Ref OperatorAccessKey
OperatorSecretKeyOutput:
Value: !GetAtt OperatorAccessKey.SecretAccessKey
Save this file as cloudformation.yaml and then create the cloudformation stack by using the following command:
aws cloudformation deploy \
--stack-name kms-example \
--template-file ./cloudformation.yaml \
--capabilities CAPABILITY_NAMED_IAM \
--region us-east-1This command will create a cloudformation stack named kms-examplethat have a user which username is kms.userand its credentials will be written in the OperatorAccessKeyOutput and secret access key in the OperatorSecretKeyOutput. You can view the credentials using AWS console or by using the following command:
aws cloudformation describe-stack kms-example
With our user in place, its time to create the KMS Key that will be used to encrypt and decrypt our secrets files.
Creating the KMS Key
Now we going to reuse the same cloudformation.yaml file, by adding the following declaration under the ‘Resources’ Key (right after OperatorAccessKey and before ‘Outputs’ Key):
KMSKey:
Type: AWS::KMS::Key
Properties:
KeyPolicy:
Version: 2012-10-17
Statement:
- Sid: Enable Root IAM User Permissions
Effect: Allow
Principal:
AWS: !Join
- ''
- - 'arn:aws:iam::'
- !Ref 'AWS::AccountId'
- ':root'
Action: 'kms:*'
Resource: '*'
- Sid: Enable User Permissions
Effect: Allow
Principal:
AWS: !Join
- ''
- - 'arn:aws:iam::'
- !Ref 'AWS::AccountId'
- ':user/kms.user'
Action:
- "kms:DescribeKey"
- "kms:Encrypt"
- "kms:Decrypt"
- "kms:ReEncrypt*"
- "kms:GenerateDataKey"
- "kms:GenerateDataKeyWithoutPlaintext"
Resource: '*'In the Ouput Section add the following values:
KMSArn:
Value:
!GetAtt KMSKey.ArnIn the snippet above we’re describing the following actions:
- Create a KMS Key where the root account can do everything and the account
kms.usercan use the it en|de|re|crypt values. - Output the Arn of KMS Key, which will be used by sops to find the Key during the operations.
Update the kms-example stack by using the same command:
aws cloudformation deploy \
--stack-name kms-example \
--template-file ./cloudformation.yaml \
--capabilities CAPABILITY_NAMED_IAM \
--region us-east-1Check the KMSArn value by using the command:
aws cloudformation describe-stack kms-example
This is the reference to the kms key expected by Mozilla SOPS in its operations.
Mozilla SOPS
Now that we have a User and a KMS Key created, its time to create and decrypt/encrypt some values. Lets start with a yaml file:
sops --kms <ARN> credentials.yamlIf all steps were executed correctly, the following text will appear (using your default editor):
Change the content to the following and save it:
---
secrets:
db:
user: username1
password: password1Now, if you open the file using an text editor, you going to see something like this output (plus some metadata used by sops):
secrets:
db:
user: ENC[AES256_GCM,data:LLTdXvf5Q60E,iv:qxUdc93ZvFlCieMrGJ+q4R9YJrr017mEdSKalp1FO1I=,tag:0n9DjDb2ikPquTm2mWI/iQ==,type:str]
password: ENC[AES256_GCM,data:btCtnukKSXyN,iv:FjO3I0umDFRiVtLsWQJ2FUtIMimWOtjaycQ6TM89uDc=,tag:ZpFQz/s/KdVABO9fL5tmkg==,type:str]If you change your mind and want to edit the file, you can use the sops command to open the editor again:
sops credentials.yamlThis time you won’t need to use the kms <ARN> parameter because its was added to the metadata in the file. Nice isn’t?
If you want to generate a decrypted version of the file, you can use the command:
sops decrypt credentials.yamlWhich outputs the content to stdout or you can redirect to another file.
With this, you can commit the file to your VCS or even send the variables to Hashicorp Vault/S3/GCS by using the SOPS options!
What else?
Using SOPS is one way on how you can leverage your VCS and workflow around it, keeping the secrets with the source code.
Mozilla SOPS can send the credentials for a hashicorp vault or even be used to export the variables to environment.
There are other solutions like Bitnami KubeSeal (to work with Kubernetes) for example, but for now, that’s all Folks!
Edit: some reddit people assert that secret as a code would have a different meaning that intended in some scenarios, so I updated the title to match the correct meaning.
