Deploy Machine Learning Models On AWS Lambda

A Hands-On Guide to Serving Machine Learning Models On AWS Lambda

Introduction :

After spending several weeks/months designing and learning an ML model, one of the hardest parts can be actually deploying and maintaining the model in a production environment where you need a server and a working environment.

AWS has made this process a lot more easier by offering to host and deploy models via AWS Lambdas, which is very cheap, and “scales” automatically by running in parallel on AWS infrastructure, and guess what, you only pay when you make predictions requests (invoke the lambdas).

In this article, we will teach you how you can leverage AWS Lambdas to deploy your machine learning / deep learning models.

Let us start.

1 — Preparing The Model:

In this section we will prepare, fit and save our model which will be used in our tutorial.

The data used in this tutorial is a real-world dataset know as the the Adult dataset available in the UCI Repo and it is the data is about predicting if the potential income a person is more than $50K/yr or it is less.

The steps we will take in this section are :

• Load and investigate the data
• Fit and save the model.

1.1- Load And Investigate The Data :

Let us load our data :

We used the shap package to load our data,

Now let us inspect it :

Adult data set

It seems all the features are set.

We will build a basic lightgbm model and save it, so there will be no preprocessing or features engineering as our main purpose of this tutorial is to deploy in AWS Lambda.

So let us do that:

1.2 — Fit And Save The Model:

import lightgbm  as lgbm
params = {'objective':'binary',
          'booster_type':'gbdt',
          'max_depth':6,
          'learning_rate':0.05,
          'metric':'auc'
    }

dtrain = lgbm.Dataset(data,labels)

model = lgbm.train(params,dtrain,num_boost_round=100,valid_sets=[dtrain])

model.save_model('saved_adult_model.txt')

As you can see, a basic model, no train/val split, basic parameters.

Now that we have our model ready, let us move to AWS, the main purpose of this tutorial.

2 — AWS Lambdas :

Before we start digging into using this service, let us first define it:

AWS Lambda is a compute service that lets you run code without provisioning or managing servers.

So what does this mean?

In simple words, it means whenever you have a ready-to-deploy machine learning model, AWS lambda will act as the server where your model will be deployed, all what you have to do is, give it the code + the dependencies, and that’s it, it is like pushing your code to a repo.

So let me show you how to do that:

First, you are going to need the serverless framework — an MIT open-source project — which will be our tools to build our App, so let us start:

The steps we will follow are these :

• Install the serverless framework
• Create a bucket in AWS
• Push our trained model to the created bucket
• Build main.py, the python file that will call our model and do predictions
• Build the serverless.yml file, in which we will tell the serverless framwork what to do (create the lambda function)
• Test what we have built locally (generating prediction with our model using the serverless framework)
• Deploy to AWS.
• Test the deployed app.

These will be the steps we are going to follow in this tutorial in order to deploy our trained model in AWS lambda.

So let us start:

Important Remark:

For the rest of the tutorial, make sure you are always in the directory where the files are, the requirements.txt, the main.py and the saved_adult_model.txt, and since I mentioned it, this is our requirements.txt:

lightgbm==2.2.3
numpy==1.17.0
scipy==1.3.0
scikit-learn==0.21.3

2.1 — Install The Serverless Framework :

To Install the serverless framwork in ubuntu, first you have to install npm. In order to do that, you can run the following commands in your terminal:

curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
sudo apt-get install nodejs

The above commands will install nodejs and also npm.

Next you can check that everything was installed correctly by running :

$ node -v

Which will return the version of nodejs

npm -v

Which will return the version of npm.

Now that we have installed npm, let us install serverless by running the following command:

npm install -g serverless

You can check that everything is installed successfully by running :

serverless

If you reached this point with no errors, then congrats, you have serverless installed and you are all set.

Let’s us move on to the next step.

2.2— Create A Bucket In AWS :

Our next step is to push the model we have trained to an AWS bucket, and for that we need first to create a bucket, so let us do that:

Creating a bucket on AWS can be done from the command line using the following code :

aws s3api create-bucket --acl private --bucket deploy-lgbm --create-bucket-configuration  LocationConstraint=eu-west-1

The above command will create a bucket called deploy-lgbm in a private mode in eu-west-1 location.

2.3 — Push Our Trained Model To The Created Bucket :

So, now that our bucket is ready, let us push our trained model to it by running the following command :

aws s3 cp saved_adult_model.txt s3://deploy-lgbm/model/saved_model.txt

Perfect, now let us move on to the next step, building our main.py python file which we will use to call our model and make predictions.

2.3 — Build The Main.py File:

When it comes to deploying in AWS lambdas, the main function of your code is a function called lambda_handler (or any other name we choose to give it, although the standard one is lambda_handler).

Now, why this function is the important one?

That function is the one AWS lambdas will execute each time you invoke it (interact with it).Thus, that function is the one will receive your input, make the prediction, and return the output.

If you have ever worked with AWS lambdas from cloud9, you will notice that when you create a new lambda function and import it, the standard definition of the lambda_function is this :

def lambda_handler(event,context):

    return {'StatusCode':200,
    'body':'Hello there, this is Lambda'}

As you can see, the lambda function expects 2 inputs — an event, and a context:

• The event will contains the information that we will send to the lambda, which will be in this case the samples we want to predict. (they will be in a json format)
• As for the context, it usually contains information about the invocation, function, and execution environment. for this tutorial we won’t be using it.

So let us summarize what we are gonna do in this section :

• First we need to get our trained model from the bucket, initialize our lightgbm and return it, so we will build a function for that.
• Then we are going to make predictions with our model, so we are going to build a function for that too.
• And finally, inside our lambda_handler function we will put all these things together, which mean, receive the event, extract the data from the event, get the model, make predictions, and then return the predictions.

So simple right?

Now let us build our file:

First we will build the get_model() function, which will download the trained lightgbm, and then initialize our model and return it :

Download the saved model.

As you can see, first we created an access to our bucket deploy-lgbm, using boto3, and then we used the method download_file to download our saved_model.txt and save it in /tmp/test_model.txt (recall that we saved the model in the bucket using the key : model/saved_model.txt).

All clear right? Let us move on then.

Now we will build the predict function, the function which will get the model, a data sample, do a prediction and then return it :

predict function

Let me explain what the above function does:

The function gets the event, extracts our data from the event, and gives the extracted data to the model to make prediction.

So simple right?

Important remarks :

• For best practice, always use json formats to pass your data in the event.
• In our case, things are sample, we extract the data and pass it to the model directly, in most other cases, there will be some processing on the data before you pass it to the model, so you will need another function for that, you will call before passing the data to the model.
• Always split your process into multiple functions, we could have put everything in the lambda function, however our code won’t be that beautiful anymore. So always use a function when you can.

Now the last step is to define our lambda handler function, so let us do that :

lambda handler

As you can see, it is a very basic function, and it will grow more complex in a real world project.

What it does is clear, get the event and send it to the predict function to get the predictions, and then return the output in the standard format (you should always use that format): a dict with a Statuscode, and the results in a body.

So, this is the end of this section, let us move on to the next on : building the serverless.yml file.

2.4 — Build The Serverless.yml File :

As we have mentioned in the start of this article, the Serverless framework will be our tool to communicate with AWS and create a lambda function which will act as the server that will host our trained model. For that we need to tell serverless all the information that it needs:

• Who is the provider? For example, is it AWS or google,
• What is the language used?
• What we want to create?
• What roles it should have ?…etc.

All of these instructions, we will pass them in the serverless.yml file, so let us start building it:

First, we will give our service a name, let us say: test-deploy

service : test-deploy

Next section in our file will be about our provider, for this case it is AWS, the instructions in the yml file will look like this:

So, what did we do in the above lines of commands? Let me explain:

• We set the name of our provider, which is aws,
• The language used (python3.6),
• the region where our lambda is going to be deployed,
• the deployment bucket that serverless will use to put the package.
• the iamRoleStatements, which mean this :
  Our lambda function is going to download the trained model from a bucket in aws, and by default, it does not have the permission to do that. So we need to give it that permission, and this is why we created a role, so we can give to our lambda the permissions it needs (for this case just the access to a bucket. In other cases could be more, you can consult aws documentation for a detailed explanation on the matter).
  And to give more example about roles, let us say that you need to invoke another lambda from this lambda, in this case this lambda needs permission for that, so you have to add them in the iamRoleStatements.

Important remarks:

• The bucket where we put our model, and the bucket used by lambda should be in the same region (for this tutorial we used eu-west-1), if they are not in the same region it won’t work.

The next section in our serverless.yml file, will be about the function we are going to create :

As you can see,

• First we define some very basic things, like the name and description.
• We define our handler :
  Recall what we said about lambda_function, we mentioned that this function will be the one doing all the work. Now this is the point where you tell serverless who is your lambda_handler function; for this case we have defined it with the name lambda_handler in our main.py file, so we put handler : main.lambda_handler. As we said earlier, we can give it what ever name we want, like for example, we can name that function hello, but then we have to put in the handler : main.hello.
• We define our event :
  How are we going to communicate with our lambda function, or in other words, how are we going to trigger (invoke) our lambda function. For this tutorial we are going to use http events, which means, invoke the lambda function by the call of a url, which will be a POST and the resource will be /predictadult.

Next Section is about Plugins:

What does that mean? Let me explain :

So far we instructed the serverless about who is our provider, and what is our function. Now for our code to work, we need the packages to be installed, and we have already put them in a requirements.txt file, so, we need to tell serverless to install those requirements, and for that we will use a Plugin called serverless-python-requirements.

We will add it to our serverless.yml file like this :

plugins:
 - serverless-python-requirements

The last thing we are going to add in our file is an optimization thing, but why we need optimizations? Let me explain :

Lambda function has some limitations for the maximum size of the package to be uploaded, and the maximum unzipped file allowed is of size 250 MB. Sometimes we exceed this amount, and to reduce it we can remove some garbage that exists in our packages which will save us some Megabytes. To do this, we instruct serverless by adding the following command in our serverless.yml file :

custom:
 pythonRequirements:
  slim : true

And that is it, the full serverless.yml file will look like this :

service : test-deploy

plugins:
 - serverless-python-requirements

provider:
 name: aws
 runtime: python3.6
 region : eu-west-1
 deploymentBucket:
  name : deploy-tenserflow
 iamRoleStatements:
  - Effect : Allow
    Action:
     - s3.GetObject
    Resource:
     - "arn:aws:s3:::deply-tenserflow/*"

custom:
 pythonRequirements:
  slim : true

functions:
 lgbm-lambda:
  name: lgbm-lambda-function
  description : deploy trained lightgbm on aws lambda using serverless
  handler : main.lambda_handler
  events :
   - http : POST /predictadult

Cool, now let us move to the next chapter: Testing what we have built locally.

2.5 — Test What We Have Built Locally :

So it is testing time:

First, your local directory should be like this :

Now that our model is ready, also as our serverless.yml file, let us invoke our serverless locally and test if everything is working by running this in command line:

serverless invoke local -f lgbm-lambda -d '{"body":[[3.900e+01, 7.000e+00, 1.300e+01, 4.000e+00, 1.000e+00, 0.000e+00,4.000e+00, 1.000e+00, 2.174e+03, 0.000e+00, 4.000e+01, 3.900e+01]]}'

If you followed the steps correctly you should get an output out of this command. In this case the output is:

{
“StatusCode”: 200,
“body”: 0.0687186340046925
}

As you can see, we choose the option invoke local, which mean we are using our computer, not the cloud, we also passed only 1 sample through the ‘body’ field (those values are the features values, not very elegant why right ?)

So, it seems everything is working locally, now let us deploy our lambda.

2.6 — Deploy To AWS:

So, it is deployment time:

Once everything is set and working, deploying a lambda is as easy as running this line of command :

serverless deploy

And that’s it, you will start seeing some log messages about the package getting pushed, you will also see the size of your zipped package.

2.7 — Test The Deployed Model :

Once the deploy command is executed with no errors, and your lambda is deployed, you will get your end point (the url) which we will use to make predictions. This url will be something like this https://xxx/predictadult :

And to test our prediction we will run this command:

curl -X POST https://xxx/predictadult -d [[3.900e+01, 7.000e+00, 1.300e+01, 4.000e+00, 1.000e+00, 0.000e+00,4.000e+00, 1.000e+00, 2.174e+03, 0.000e+00, 4.000e+01, 3.900e+01]]

And that’s it, Congrats, you have deployed your model in a AWS lambdas function and now can serve you.

If you faced any error while re-running the above tutorial, you can reach out to me, my contact info are below, Iwill be very happy to help.

I hope you found this tutorial very insightful and practical, and I hope you are feeling now a better Data scientist after reading all these words.

Until next time, meanwhile if you have any questions for me, or you have a suggestion for a future tutorial my contact info are below.

About Me

I am the Principal Data Scientist @ Clever Ecommerce Inc, we help businesses to Create and manage there Google Ads campaigns with a powerful technology based on Artificial Intelligence.

You can reach out to me on Linked In or by gmail: errabia.oussama@gmail.com.