AWS Lambda + Serverless Framework + Python — A Step By Step Tutorial — Part 2 “Using AWS KMS with Lambda to Store & Read Sensitive Data & Secrets”

Using serverless technologies is becoming more and more mainstream. Serverless may make your life easier in several contexts, however, you are always responsible for securing your code. As a developer, one of the things you need to know is how to store secrets safely. Just google “Github leaks”, and you will find how easy is finding logins, passwords and other sensitive information.

Aymen El Amri
Sep 21, 2018 · 8 min read

Disclaimer

This content is part of / inspired by one of our online courses/training. We are offering up to 80% OFF on these materials, during the Black Friday 2019.

You can receive your discount here.



In this tutorial, you are going to learn how to use AWS services in order to deploy a serverless function without versioning your secret information.

You will learn how to use AWS KMS/SSM and invoke a secret from your Lambda code.

Don’t forget to subscribe to Shipped: An Independent Newsletter Focused On Serverless, Containers, FaaS & Other Interesting Stuff and take a look at Practical AWS, a training concerned with the actual use of AWS rather than with theory & ideas.

Prerequisites

The first thing you need to do, especially if you are starting learning AWS Lambda and the Serverless Framework, is following the first tutorial.

If you don’t want to use your AWS default profile, you can use a different profile before starting this tutorial.

Example:

You can also add the default AWS region.

Note that your profiles can be found in the file:

As said in the part 1 of this tutorial, if you don’t have a credential file, you can always configure one using:

Example:

If you already have a credential file, you can choose one of the profiles to use, each time you will execute a serverless command.

Example:

Let’s create a folder called using-kms and activate the virtual environment.

Initializing our Project

In the app folder, we need to create a serverless.yml file. This file is needed to configure how our application will behave.

We need also to create our Python function (called handler.py):

In order to do this, let’s execute this command:

This will create the handler file, the configuration file and the .gitignore file:

This is how our serverless.yml file looks like:

This is how the handler.py file looks like:

Deploying our Project

Once the template files are created, we have a working AWS Lambda function, we need to deploy it:

Note: You need to change the profile name to use your own one.

The deployment output looks like this. You can see that our code is zipped and deployed to a S3 bucket before being deployed to Lambda.

The function is now deployed.

In the remaining parts, we will create secrets using AWS KMS, then call these secrets from our serverless code.

Using AWS KMS/SSM

This is how AWS defines its service:

AWS Key Management Service (AWS KMS) is a service that combines secure, highly available hardware and software to provide a key management system scaled for the cloud. AWS KMS uses customer master keys (CMKs) to encrypt your Amazon S3 objects.

Let’s start by creating a key to use later:

Output:

This is an example of using AWS Secure Secrets Manager with the previously generated key:

Output:

With AWS Systems Manager Parameter Store, you can create Secure String parameters, which are parameters that have a plaintext parameter name and an encrypted parameter value. Parameter Store uses AWS KMS to encrypt and decrypt the parameter values of Secure String parameters

After creating the secret it will be encrypted using the generated key and we can view our secret using:

Output:

You can view the decrypted secret using:

Output:

Invoking Secrets from the Lambda Function

In order to use the secret that we created previously, we are going to edit the file handler.py in order to add this code:

We should, of course, have a valid serverless.yml file. This is mine for the above function:

(Note that we are including the AWS key pair in our serverless.yml file which is not a good practice unless your files is not versioned and protected. We don’t prefer to complicate things for you right now but it could be a good practice to encrypt variables like ACCESS_KEY in the environment (build/test/prod/dev). This practice is more related to CI/CD and less to Serverless framework. Another way to configure your credential without using them in your yml file is using serverless config command:serverless config credentials --provider aws --key <ACCESS KEY ID> --secret <SECRET KEY> )

We can now deploy and visit the generated URL:

As you can notice in the output above, this is the URL we can visit to get the secret:

This is what our function return for now:

In order to extract only the password value from this output, we can use

Tailing Logs

Sometimes, you need to see the execution or the deployment logs. This is the command to tail the logs:

Deleting the Function

Using sls remove command, we can delete the deployed service from AWS Lambda.

We can add other options like:

  • --stage or -s The name of the stage in service.
  • --region or -r The name of the region in stage.
  • --verbose or -v Shows all stack events during deployment.

Note: Removing a service will also remove the S3 bucket.

Connect Deeper

In the first part of this tutorial, we have seen how to deploy our first Lambda function using the Serverless Framework.

This part (2) will help you to store and use secrets from your AWS Lambda function.

Stay in touch as the upcoming posts will go more in depth.

I am creating a series of blog posts to help you develop, deploy and run (mostly) Python applications on AWS Lambda using Serverless Framework.

You can find my other articles about the same topic but using other frameworks like Creating a Serverless Uptime Monitor & Getting Alerted by SMS — Lambda, Zappa & Python or Creating a Serverless Python API Using AWS Lambda & Chalice

Don’t forget to subscribe to Shipped: An Independent Newsletter Focused On Serverless, Containers, FaaS & Other Interesting Stuff.

You may be interested in learning more about Lambda and other AWS service, so please take a look at Practical AWS, a training concerned with the actual use of AWS rather than with theory & ideas.

FAUN

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts

Aymen El Amri

Written by

Aymen El Amri is the founder and CEO of www.eralabs.io and www.faun.dev community. He is a tech author, cloud-native architect, entrepreneur and startup advisor

FAUN

FAUN

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts. Medium’s largest DevOps publication.

Aymen El Amri

Written by

Aymen El Amri is the founder and CEO of www.eralabs.io and www.faun.dev community. He is a tech author, cloud-native architect, entrepreneur and startup advisor

FAUN

FAUN

The Must-Read Publication for Aspiring Developers & DevOps Enthusiasts. Medium’s largest DevOps publication.