Adding Pandas library to AWS Lambda Layer using Docker

Pallavi Naidu
Plumbers Of Data Science
2 min readSep 17, 2021

Prerequisite
1. Docker should be installed on your local machine.

2. The below mentioned steps are for python 3.8 and Mac OS. It can be run on a windows machine as well.

Step 1: Create a requirements.txt file and name the python libraries you wish to import. Create a folder and put the requirements.txt file into it.

pandas
pytz

Step 2: On the terminal, navigate to the folder where the requirements.txt file resides. Run the below mentioned docker command.

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"

What happens here: The docker image of python 3.8 is pulled from lambic/lambda locally to the current folder. The libraries mentioned in the requirements.txt file are imported using the pip install command.
After the build is completed, you should be able to see the following structure in your current folder.

Step 3: Zip the python file to be uploaded in Lambda layers. Run the below mentioned command in your terminal.

zip -r mypandaslayer.zip python > /dev/null

Step 4: Go to the AWS lambda console and click on Layers in the left pane. Click on Create layer button on top right.

Step 5: Fill in the details such as Name and description. Upload the zip file created in step 3 and hit upload button. Hit the Create button at the right bottom corner after all the details are filled.

Step 6: Pandas library needs Numpy too to function. AWS has embedded the numpy library in lambda layer which can be directly used.
To add Numpy: Go to lambda function -> click on layers -> Add layer

In the above screen, choose AWS layers.

From the drop down menu choose AWSLambda-Python38-SciPy1x and the version number. Then click on Add.

--

--