AWS Lambda Layer for Python3.9 Snowflake Connector

Introduction:

Executing SQL queries against Snowflake from an AWS Lambda function using Python3.9 may lead to encountering a comparable error message as shown below.

⛔️ No module named ‘snowflake’

Error message

Worry not, as there’s a straightforward solution you can implement quickly. In this article, explore how to resolve this issue using an AWS Lambda Layer.

Overview:

Python packages containing compiled code for the Snowflake connector may not always be compatible with Lambda runtimes out of the box, especially on non-Linux operating systems. To resolve compatibility issues, install the below mentioned packages using pip with — platform parameter set to manylinux2014.

To overcome this issue using an AWS Lambda Layer, Follow the next three steps.

Step 1: Install Linux operating system and Python3.9.

Prior to proceeding, ensure your local machine runs on a Linux operating system and has Python3.9 installed. If not, install Linux and Python3.9 first. These prerequisites are vital for creating a Lambda Layer-compatible deployment package.

Step 2: Choose the Appropriate Architecture.

To install the Snowflake Connector Package use the appropriate architecture-specific pip command.

▹ For “X86_64” architecture:

pip install \
    --platform manylinux2014_x86_64 \
    --target=python/lib/python3.9/site-packages \
    --implementation cp \
    --python-version 3.9 \
    --only-binary=:all: --upgrade \
    snowflake-connector-python==3.2.1

▹ For “arm64” architecture:

pip install \
    --platform manylinux2014_aarch64 \
    --target=python/lib/python3.9/site-packages \
    --implementation cp \
    --python-version 3.9 \
    --only-binary=:all: --upgrade \
    snowflake-connector-python==3.2.1

Step 3: Create AWS Lambda Layer.

• When running the above pip command, a “python” folder is created.

• Open the folder in the file explorer and compress the folder as a zip file.

Create a zip File

• Create a layer in AWS Lambda and upload the ZIP file.
• Choose the corresponding architecture and runtime (Python 3.9) for the Lambda Layer, matching the selections made in the pip command.

Create layer

• Add the Layer to your Lambda function and execute your function.

Add layer

Execution Succeeded

Conclusion:

By following these steps and leveraging AWS Lambda Layers, you can ensure that Python-based Snowflake connectors seamlessly integrate with AWS Lambda functions, regardless of the underlying architecture. This approach simplifies deployment and enhances compatibility, enabling smooth execution of SQL queries against Snowflake within Lambda functions.

About Us:

Bi3 has been recognized for being one of the fastest-growing companies in Australia. Our team has delivered substantial and complex projects for some of the largest organizations around the globe, and we’re quickly building a brand that is well-known for superior delivery.

Website: https://bi3technologies.com/

Follow us on,
LinkedIn: https://www.linkedin.com/company/bi3technologies
Instagram: https://www.instagram.com/bi3technologies/
Twitter: https://twitter.com/Bi3Technologies