Automated Lambda Function Generator for Python

Deploying an AWS Lambda is a very easy and quick process. All you need is a python file with a function, and voila, you are ready to go. That is, unless, you have dependencies on packages with FFI invocations. Since the python code is running in a AWS python sandbox, you can run into problems if you generate the site-packages on a different platform.

Quick refresher, the lambda function lives inside /var/task folder and if you want to do anything different, you need bundle all your dependencies into that folder, aka: pip install newspaper3k --target /var/task/ and generate a zip artefact which you can deploy as a lambda function.

You could spin up a AWS EC2 instance and using virtualenv generate the package. However, this is not quick in build environment or can be built and tested locally. I found this amazing library of docker containers that simulate the AWS Lambda.

Let’s usedocker-compose to generate the python site-packages inside /var/task along withall the libraries and it’s sym-links. This will work both lambci docker container and also AWS.

Let’s take a look at a simple lambda_function.py.

And this function requires a python package with system dependencies — newspaper3k. You can now use docker-compose and an installation script to build this artifact inside docker.

docker compose file

artefact-generator script

I noticed that some of the docker images in lambci repository don’t ship with zip package, so I have a wrapper script that runs locally on the machine runs docker-compose and zip the artefact.

You can generate the package by executingbash package.bashand you will see a package/lambda_function.zip file . You can now ship this package in your AWS Lambda and voila should work without any issues.

After going through this, I wonder if there is already a packager that’s out there that does this already, any thoughts?