Deploying deep learning models on AWS lambda
About AWS Lambda
AWS Lambda is a serverless computing service provided by Amazon Web Services. The definition of serverless architecture is — it is a stateless compute container designed for event-driven solutions just like microservice architecture where monolithic applications are broken into simple smaller services which are easy to code, manage and scale.
AWS Lambda Limits
AWS Lamda poses certain restriction on the amount of storage and computation that we can use for Lambda functions. Each function executes within a 64-bit Amazon Linux AMI and the size specs are listed below:
For the complete list of resource lists, please visit https://docs.aws.amazon.com/lambda/latest/dg/limits.html
Deploying a deep learning model using AWS Lambda
It is quite challenging to deploy deep learning models in AWS lambda because of various factors. a) The size of the model which can differ based on the architecture chosen b) Essential packages and dependencies to efficiently run the model which includes libraries for preprocessing the data etc.,
For eg., an image classification model might require essential packages such as the deep learning framework of choice (e.g., TensorFlow/Keras/mxnet, etc.,) and some preprocessing packages such as OpenCV or Pillow.
In simple terms, a deployment package would consist of programs, packages, and models. We will list some of the issues we faced while trying to deploy an image classification model. Our model had the following dependencies:
Tensorflow == 1.8.0
Keras == 2.2.4
Numpy == 1.16
Pillow == 5.4.1
Issue 1: Trying Lamda deployment for the first time we packaged everything in a zip file, and the size of the deployment package was more than 500 MB which is off the limits, and there is no way to deploy this using a Lambda function.
Possible Solution: We started exploring ways which could lead us to deploy our service using AWS Lambda. While going through AWS Lambda’s developer guide something that caught our attention was the AWS Lambda layers. Lambda layers are custom layers which contain libraries, a custom runtime, and other dependencies and each layer is a zip file containing libraries and dependencies. For eg., It is possible to create a layer which contains Scikit-learn and use it as the base layer for multiple lambda functions. AWS Lambda layers are very flexible, and it allows for various operations such as stacking layers, merging layers, etc. Please visit https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html for more details
Issue 2: There are certainly ready to use lambda layers in each AWS region identified by its ARNs (Amazon Resource Names) which encompasses libraries and certain other dependencies. For eg., the layer we identified had Tensorflow, Pillow, Numpy, etc .,. The size of the layer is around 230 MB. There is not enough space to accommodate the model, which is close to 100 MB.
Possible Solution: There is ephemeral storage of 512 MB in the /tmp directory. Rather than having the model as a part of the deployment package, we can load the model to an s3 bucket and download it from the s3 bucket to the /tmp directory. Each function provides 512 MB of additional space, so during the start of the program, we can download the model to the /tmp directory and load it from there.
In the next part of the article, we will demonstrate how to deploy multiple models on a single Lambda function, creating deployment packages and creating custom lambda layers.
Masala.AI
The Mindboard Data Science Team explores cutting-edge technologies in innovative ways to provide original solutions, including the Masala.AI product line. Masala provides media content rating services such as vRate, a browser extension that detects and blocks mature content with custom sensitivity settings. The vRate browser extension is available for download via the Chrome Web Store. Check out www.masala.ai for more info.
