How to Create a Python Layer in Aws Lambda

In this article I will describe how to create a layers with libraries or custom codes and share between all your lambdas

Once you need to use a python library you have two options:

1. Deploy your code alongside with necessary libraries. (will result in large files to upload)
2. Create layers and reference them in your lambda function

Before We Start

The intent of this article is to be very practical. If you want a deep discussion about when to use and performance questions will list some articles at the end.

I will create a sample using pandas, once this library has other dependencies and could be a good sample.

Let’s get our hand dirt:

1. Install the Library Locally

In your terminal create a folder called ‘python’

mkdir python
cd python

Once done, install locally your library:

pip3 install pandas -t .

This is necessary because lambda layer need to follow a specific structure, that is detailed here

Your folder will be similar like that

Folder Structure

Note that others libraries like numpy is included and must be added to the final package

2. Create Zip File

Zip this folder using this code on parent folder:

zip -r pandas_layer.zip .

*** Your zip file should have a folder named python with all content inside

3. Create your Layer on AWS

Upload your pandas_layer.zip to your s3 bucket

Navigate to Lambda > Layers and create a layer. Don`t forget to reference the compatibles runtimes, this will help you to add easily your layer to your function.

I`ve used this code as sample:

And if I run, finally I’ve get this result:

Success

Here I`ll let some articles that discuss in details performance and best usage of lambdas layers and lambda docs

https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
https://lumigo.io/blog/lambda-layers-when-to-use-it/
https://www.slideshare.net/AmazonWebServices/reinvent-deep-dive-on-lambda-layers-and-runtime-api
https://medium.com/consulner/performance-of-aws-lambda-with-and-without-layers-9bffbb5434f3