AWS Lambda Layers for Python

Image for post
Image for post

Amazon Lambda is probably the most famous serverless AWS service. Generally, It offers a low-cost solution and a relatively simple and straightforward platform for implementing functions in different languages. If your script is not complex, setup is incredibly simple. But, One of the main problems with Lambda is that it becomes tricky to set up as soon as your functions and triggers get more complex.

If you were using the Docker platform, you would import your custom packages with PIP. Unfortunately, there is no docker and PIP package manager. There are just built-in packages available. So the AWS found a solution named Lambda Layers. The solution is to give a zip file that included external python libraries to AWS. You can still use the PIP command to prepare this package file, just with a small extra parameter.

export OUTDIR=/home/batur/layers
pip3 install -r requirements.txt -t
$OUTDIR/python/lib/python3.8/site-packages/

“-t” is a target directory parameter. We used it because we don’t want to add all old development packages to our layer file. We added just needed libraries to the “requirements.txt” file. Otherwise, probably we will produce a huge unnecessary file.

For example: (requirements.txt)

requests==2.21.0
schedule==0.6.0
paramiko==2.6.0
psycopg2
pytz

After finishing the pip command, you can zip them.

cd $OUTDIR
zip -r layer-packages.zip .

After creating a zip file, you can send your S3 bucket.

export S3_BUCKET=my_lambda_layer
aws s3 cp layer-packages.zip s3://${S3_BUCKET}/

That is it! Anyway, I have a surprise for you. I created a docker system to do all process in a one-liner.

https://github.com/baturorkun/aws-lambda-layers

You can find all information in the README file.

After creating the layer you can add it to your Lambda function. There are two methods to do it. Using CLI or UI/Web Console. If using the web console is enough for you, it is very simple.

At first, you must create a layer on AWS Lambda. You will see the “Additional resources” menu on the left side and the “Layers” link. After the “Create layer” button, will see this page.

Image for post
Image for post

You can upload your zip file from local or use the s3 bucket link. After creating a layer, you can use it on functions.

When you opened your function page, you can see the Layers tab and click here.

Image for post
Image for post

After clicking the “Add layer” button, chose Custom layers and select your layer name from the select box.

Now, you can import external packages and use it in your codes on this lambda function.

Written by

DevOps & Software & Architect & Linux Geek — http://baturorkun.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store