A predictive engine API deployment with AWS and serverless in minutes.

Photo by Mika Baumeister on Unsplash


I worked a couple of weeks with AWS Sagemaker vs. AWS Lambda and came to the conclusion that AWS Sagemaker has some drawbacks.

  • Every model has one endpoint with a running instance ( You have the ability using your own docker container if you would otherwise, but this is not really handy)
  • The deployment process demands a lot of configuration.
  • You pay for your endpoint instance as soon the endpoint is running. If the endpoint is 20h/day idle it makes no difference.

On the contrary you have the ability to deploy a machine learning model with AWS Lambda , API Gateway and serverless fast and the freedom to do anything as long as Lambda support your needs. Layer support in AWS Lambda and serverless makes it even more easy.

Deployment in 4 steps

  1. First of all, let’s build a classification model and serialize it.
import pandas as pd
import numpy as np
import pickle
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
pd.set_option('display.float_format', lambda x: '%.5f' % x)
data = load_wine()
df = pd.concat([pd.DataFrame(data.data), pd.DataFrame(data.target)],ignore_index=True,axis=1)
df = df.sample(frac=1)
X_train, X_test, y_train, y_test = train_test_split(df.iloc[:,:-1],df.iloc[:,-1],test_size=0.33, random_state=42)
logreg = LogisticRegression(C=1e5, solver='lbfgs', multi_class='multinomial', max_iter=10000)
model = logreg.fit(X_train, y_train)

2. Because AWS Lambda didn’t support scikit-learn, you need to prepare and add a layer with scikit-learn

$ PY_DIR='build/python/lib/python3.6/site-packages'
$ mkdir -p $PY_DIR
$ pip install -r requirements_aws.txt -t $PY_DIR

3. Now create the AWS lambda function get_prediction.py

import json
import pickle
import logging
import numpy as np
from sklearn.linear_model import LogisticRegression
logger = logging.getLogger()
model = pickle.load(open("model/wine_model.pkl", "rb"))
def handler(event, context):
payload = event.get("body")
data = json.loads(payload).get("data")
pred = list(model.predict_proba(np.array(data).reshape(1, -1))[0])
return {"statusCode": 200, "body": json.dumps(str(pred))}

4. Last but not least, we need to specify the the service and events in our serverless.yml. This is just an example and see https://serverless.com/ for details.

service: aws-simple-predictive-engine
individually: true
name: aws
runtime: python3.6
timeout: 3
- ${self:provider.stage}-predictive-engine
limit: 1000000
offset: 0
period: DAY
burstLimit: 10000
region: eu-central-1
endpointType: REGIONAL
path: build
description: scikit-learn
- python3.6
handler: get_prediction.handler
description: This function predicts according to new data
memorySize: 256
timeout: 3
reservedConcurrency: 10
- build/**
- venv/**
- http:
path: v1/predict
method: post
private: true
- {Ref: ScikitLayerLambdaLayer}

Now we just deploy with our AWS credentials.

$ sls deploy

The deploy output from serverless contains your API_KEY and API_ENDPOINT.

Now , let’s test our endpoint

In the last step we generate some data (dumb random data) and post requests to call the endpoint.

import json
import requests
headers = {
"Content-type": "application/json",
"x-api-key": "YourKey",
endpoint = "https://YourEndpoint/dev/v1/predict"

def call_api_gateway(input_data, headers=headers,endpoint=endpoint):
input_data = {'data':list(input_data)}
r = requests.post(endpoint, data=json.dumps(input_data), headers=headers)
response = r.json()
scores = json.loads(response)
except Exception as e:
scores = [None]*3
return scores
df_scores = pd.DataFrame([call_api_gateway(v) for v in pd.DataFrame(np.random.randn(1000,13)).values])
df_scores.columns =[‘class_0’, ‘class_1’,’class_2']


If you’re finished just drop the service

$ sls remove

Conclusion and possible next steps

In my opinion AWS Lambda and serverless gives you a lot of possibilities to deploy a predictive engine API fast. Database Integration and bulk scoring capabilities might be the next interesting steps.