ML model deployment in AWS Lambda: ECR update
A third - and probably last - version of how to fit ML models and their dependencies into AWS lambda functions. What used to be a horrible experience has now become a walk in the park.
The last blog suggested mounting an AWS EFS system onto the lambda functions to easily store all the required python dependencies. But unfortunately, we noticed that in practice this approach doesn’t scale properly monetary-wise.
Currently, we deploy our models using the AWS Elastic Container Registry (ECR) service. This is an easily scalable way to deploy docker containers into the cloud. You pay for storing the docker image and per time you use it at the exact cost as an AWS lambda function. Nothing out of the ordinary.
Once the ECR is configured, all the developer needs to do is to create a docker file and that’s it! The application is ready for deployment.
Below you can find a docker file example. To keep the image lean, a multi-staged build is recommended. This can be done by configuring an as BUILD and as RUN section in your docker file.
FROM python:3.9 AS BUILD
COPY src/requirements.txt lambdarequirements.txt# install requirments and remove unnecessary files
RUN pip install --upgrade pip && \
pip install -t /var/task awslambdaric && \
pip install -r lambdarequirements.txt -t /var/task && \
for f in $(find /var/task -name \*.pth); do sed -i 's/;/;\r/g' $f; done && \
find /var/task -type d -name "tests" -exec rm -rf {} \+ && \
find /var/task -type d -name "__pycache__" -exec rm -rf {} \+ &&
find /var/task -type d -name \*.pyc -exec rm -rf {} \+ && \
rm -rf /var/task/{*.egg-info,*.dist-info} /var/task/{app.py,requirements.txt,lambdarequirements.txt,__init__.py}
FROM python:3.9 AS RUN
COPY --from=BUILD /var/task /var/task/
ENV PATH="/var/task:${PATH}"
WORKDIR /var/task
COPY src/modelling /var/task/modelling
COPY src/main.py /var/task/main.py
ENTRYPOINT ["/usr/local/bin/python", "-m", "awslambdaric"]
CMD ["main.lambda_handler"]
Our system currently processes millions of runs per month, without any issues or unexpected costs. So I highly recommend the AWS ECR system for deploying your machine learning application.
