A predictive engine API deployment with AWS and serverless in minutes.

Christian Schulz
Feb 23, 2019 · 3 min read
Image for post
Image for post
Photo by Mika Baumeister on Unsplash


  • Every model has one endpoint with a running instance ( You have the ability using your own docker container if you would otherwise, but this is not really handy)
  • The deployment process demands a lot of configuration.
  • You pay for your endpoint instance as soon the endpoint is running. If the endpoint is 20h/day idle it makes no difference.

On the contrary you have the ability to deploy a machine learning model with AWS Lambda , API Gateway and serverless fast and the freedom to do anything as long as Lambda support your needs. Layer support in AWS Lambda and serverless makes it even more easy.

Image for post
Image for post

Deployment in 4 steps

import pandas as pd
import numpy as np
import pickle
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
pd.set_option('display.float_format', lambda x: '%.5f' % x)
data = load_wine()
df = pd.concat([pd.DataFrame(data.data), pd.DataFrame(data.target)],ignore_index=True,axis=1)
df = df.sample(frac=1)
X_train, X_test, y_train, y_test = train_test_split(df.iloc[:,:-1],df.iloc[:,-1],test_size=0.33, random_state=42)
logreg = LogisticRegression(C=1e5, solver='lbfgs', multi_class='multinomial', max_iter=10000)
model = logreg.fit(X_train, y_train)

2. Because AWS Lambda didn’t support scikit-learn, you need to prepare and add a layer with scikit-learn

$ PY_DIR='build/python/lib/python3.6/site-packages'
$ mkdir -p $PY_DIR
$ pip install -r requirements_aws.txt -t $PY_DIR

3. Now create the AWS lambda function get_prediction.py

import json
import pickle
import logging
import numpy as np
from sklearn.linear_model import LogisticRegression
logger = logging.getLogger()
model = pickle.load(open("model/wine_model.pkl", "rb"))def handler(event, context):
payload = event.get("body")
data = json.loads(payload).get("data")
pred = list(model.predict_proba(np.array(data).reshape(1, -1))[0])
return {"statusCode": 200, "body": json.dumps(str(pred))}

4. Last but not least, we need to specify the the service and events in our serverless.yml. This is just an example and see https://serverless.com/ for details.

service: aws-simple-predictive-enginepackage:
individually: true
name: aws
runtime: python3.6
timeout: 3
- ${self:provider.stage}-predictive-engine
limit: 1000000
offset: 0
period: DAY
burstLimit: 10000
region: eu-central-1
endpointType: REGIONAL
path: build
description: scikit-learn
- python3.6
handler: get_prediction.handler
description: This function predicts according to new data
memorySize: 256
timeout: 3
reservedConcurrency: 10
- build/**
- venv/**
- http:
path: v1/predict
method: post
private: true
- {Ref: ScikitLayerLambdaLayer}

Now we just deploy with our AWS credentials.

$ sls deploy

The deploy output from serverless contains your API_KEY and API_ENDPOINT.

Now , let’s test our endpoint

import json
import requests
headers = {
"Content-type": "application/json",
"x-api-key": "YourKey",
endpoint = "https://YourEndpoint/dev/v1/predict"

def call_api_gateway(input_data, headers=headers,endpoint=endpoint):
input_data = {'data':list(input_data)}
r = requests.post(endpoint, data=json.dumps(input_data), headers=headers)
response = r.json()
scores = json.loads(response)
except Exception as e:
scores = [None]*3
return scores
df_scores = pd.DataFrame([call_api_gateway(v) for v in pd.DataFrame(np.random.randn(1000,13)).values])
df_scores.columns =[‘class_0’, ‘class_1’,’class_2']
Image for post
Image for post


$ sls remove

Conclusion and possible next steps


Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store