Import Custom Python Packages on AWS Lambda Function

This tutorial is for “how to import the custom (BeatifulSoup4)python packages on AWS lambda”

Step 1:

Create the virtualenv in your local machine.
Note:
The AWS lambda only support the python2.7 and python3.6. So we can create either python2.7 or python3.6 environment. Now i have python3.6, so i am using python3.6 environment. If you dont have python3.6 i will explain how to implement that too.

Install virtualenvironment in your local machine(pip or pip3).

sudo pip install virtualenv

Python2.7(If you have):

Create the virtual environment

#Go Home
manivannan@manivannan:~$ cd $HOME
manivannan@manivannan:~$ virtualenv -p /usr/bin/python2.7 BSoup
# The BSoup is name of virtualenvironment
# Activate your virtual environment using below command
manivannan@manivannan:~$ source BSoup/bin/activate
(BSoup) manivannan@manivannan:~$
# This environment only working with python2.7

Python3.6(If you have):

Create the virtual environment

#Go Home
manivannan@manivannan:~$ cd $HOME
manivannan@manivannan:~$ virtualenv -p /usr/bin/python3.6 BSoup
# The last BSoup is name of virtualenvironment
# Activate your virtual environment using below command
manivannan@manivannan:~$ source BSoup/bin/activate
(BSoup) manivannan@manivannan:~$
# This environment only working with python3.6

Successfully created the virtual environment.

Step 2:

Install Packages to Environment

pip install beautifulsoup4
pip install lxml

Step3:

Create the lambda_function.py and paste the below lines.

import json
import bs4 as bs
from urllib import request
def lambda_handler(event, context):
# TODO implement
html_data = request.urlopen("https://pythonprogramming.net/introduction-scraping-parsing-beautiful-soup-tutorial/").read()
soup = bs.BeautifulSoup(html_data,'lxml')
title = soup.find('h2')
return {
"statusCode": 200,
"body": json.dumps(title.text)
}

Step4:

Move the lambda_function.py to your environment(BSoup) site-packages folder

(BSoup) manivannan@manivannan:~$ mv lambda_function.py BSoup/lib/python3.6/site-packages/
(BSoup) manivannan@manivannan:~$ cd BSoup/lib/python3.6/site-packages/
(BSoup) manivannan@manivannan:~/BSoup/lib/python3.6/site-packages$ ls

Step5:

Take a zip to entire files inside the site-packages folder.

The lambda_function.zip file is created and located on site-packages folder.

Step6:

Create the lambda function as python3.6 and deploy the zip file to it.

Step7:

Output:

Important:

This is my zip file link and you can extract the zip file. only edit the lambda_function.py and again zip it to deploy.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade