Deploying AWS Lambda Functions for Machine Learning Workloads
AWS Lambda is a serverless and event-driven computing service in Amazon’s cloud computing offerings. Lambda functions enable organizations to run application code without managing servers and can automatically scale to hundreds of thousands of executions, within account limits.
Lambda functions can be deployed in two different ways; using a zip file or using a Lambda-compatible container image. At Insider, our default approach is to use zip file deployment packages. Using a zip file is easier to manage and faster to deploy due to less operational overhead. We use Github actions for creating and uploading the zip file on branch merge to deploy the new application code.
However, zip file archives have a 250 MB unzipped limit for the package size, including layers and custom runtimes. The limit is mostly fine for standard backend applications, but it isalmost impossible to achieve if you want to run a machine learning workload. Simply installing libraries like NumPy, SciPy, and Pandas can cause the package size to exceed the limit, let alone including a deep learning framework such as PyTorch.
The container image deployment package is the solution for these situations. One of the reasons that container image support was introduced was the increasing amount of machine learning and data-intensive workloads. AWS Lambda supports container images up to 10 GB, which is significantly larger than the constraints of the zip file deployment package.
The container images must implement Lambda runtime API to work with Lambda. To facilitate this, AWS provides base images for several runtimes. For instance, to deploy a Python 3.10 application, one can simply use the Python 3.10 base image, add the application code and dependencies, and the image is ready. Below is a minimal Dockerfile to demonstrate how to set up such an image.
FROM public.ecr.aws/lambda/python:3.10
COPY src/ ${LAMBDA_TASK_ROOT}/src/.
COPY requirements.txt ${LAMBDA_TASK_ROOT}/requirements.txt
RUN pip install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"
CMD [ "src/lambda_function.lambda_handler" ]Note that a container image can only be compatible with a single architecture. This means that if you build the image on a computer with an x86 architecture, it should be deployed to an x86-based Lambda function. And if you are building on an ARM-based instance such as Graviton or an Apple Silicon Mac computer, the image must be deployed to an ARM-based Lambda function.
When there is a mismatch between the instruction set of the Lambda function and the image, you may encounter an exception that isn’t very descriptive like the one below. This caused us to spend a considerable amount of time figuring out that the issue was due to the misconfigured architecture. So, pay attention to configuring the build server and the Lambda function architecture accordingly. Remember that you can always change the instruction set of a Lambda function atany time.
Error: fork/exec /lambda-entrypoint.sh: exec format error
Runtime.InvalidEntrypointFinally, using a Docker image instead of a zip file changes our CI/CD pipeline. We use AWS CodeBuild to build the Docker image and push it to the Amazon ECR repository for the build phase. Then, using the AWS CLI, we update the function’s image URI to the new image to complete the deployment phase.
AWS Lambda functions are incredibly useful when you want to avoid provisioning and managing infrastructure while scaling rapidly. Using zip file deployment is straightforward and quick for standard applications. However, if your use case involves packages that exceed the 250 MB limit, it’s necessary to use the container image deployment, which supports packages up to 10 GB in size. Note that the deployment type must be decided before creating the Lambda function, as it cannot be changed later on.
Don’t forget to follow Insider Engineering Blog to read similar articles. You can also find more articles about AWS Lambda below.
