Online predictions API using scikit-learn and Cloud Machine Learning Engine on GCP

Jose Miguel Arrieta
May 28 · 5 min read

Sometimes Data Scientist needs to expose their Machine Learning models to be consumed by other members of the team (e.g Back-ends developers) or directly by final users. In these cases implementing an API is useful.

This post will explain steps to train a model, store classifier on Google Cloud Storage and use Cloud Machine Learning to create an API in order to perform online predictions. The classifier will be trained using iris flower data set which consists of 3 different types of irises (Setosa, Versicolour, and Virginica). The rows being the samples and the columns being features: sepal length, sepal width, petal length, and petal width. Scikit-learn library will be used for machine-learning algorithms.

Full code and data are on Github



  • Create Bucket on Cloud Storage (and some folders) and save CSV data inside.

1. Train

The goal is to use CSV data on Google Data Storage to train a model (this could be done interactively via Datalab or in a training application package using Cloud ML) and save classifier in a Bucket.

Option 1. Using Cloud Datalab

Cloud Datalab is an easy to use interactive tool for data exploration, analysis, visualization and machine learning.

You will need to open Cloud Datalab, download CSV file from Cloud Storage, read CSV file into a pandas dataframe, train model and export classifier to Cloud Storage Bucket.

Code is explained in the following Jupyter Notebook.

Option 2. Using Cloud Machine Learning Engine

You can scale up model training by using the Cloud ML Engine training service in a serverless environment within GCP

  • Create a training application package

Open cloud shell and type

mkdir iris_training
touch iris_training/
touch iris_training/ have the following code:

  • Submit a training job
JOB_NAME=iris_training_$(date +"%Y%m%d_%H%M%S")
gcloud ai-platform jobs submit training $JOB_NAME \
--job-dir $JOB_DIR \
--package-path $TRAINING_PACKAGE_PATH \
--module-name $MAIN_TRAINER_MODULE \
--region $REGION \
--runtime-version=$RUNTIME_VERSION \
--python-version=$PYTHON_VERSION \
--scale-tier $SCALE_TIER
  • View your training logs
gcloud ai-platform jobs stream-logs $JOB_NAME

2. Deploy

The AI Platform online prediction service manages computing resources in the cloud to run your models. Serverless, fully managed hosting that responds in real time with high availability. Our global prediction platform automatically scales to adjust to any throughput. It provides a secure web endpoint to integrate ML into your applications.

The goal is to use the classifier saved in Google Cloud Storage Bucket and create an API on Cloud Machine Learning Engine, perform requests and receive online predictions. The classifier trained from Datalab Jupyter Notebook is saved inside a folder on the previous bucket I created gs://demo-iris/model/model.joblib

Deploy models and versions

AI Platform organizes your trained models using model and version resources. An AI Platform model is a container for the versions of your machine learning model.

Create a model resource

Open a cloud shell and type the following:

gcloud ai-platform models create "iris"

Set Environment Variables


MODEL_DIR refers to the path where the model was saved, not the model itself.

Create a version

gcloud ai-platform versions create $VERSION_NAME \
--model $MODEL_NAME \
--origin $MODEL_DIR \
--runtime-version=1.13 \
--framework $FRAMEWORK \

AI Platform uses images, based on runtime version designation, to configure cloud resources to service your training and prediction requests.

Get Information about your new version using:

gcloud ai-platform versions describe $VERSION_NAME \
--model $MODEL_NAME

Also, you can check the list of models in the AI Platform dashboard.

3. Test API

The online predictions will be tested in 3 ways, from inside Google Cloud, using curl and using Postman.

Send a request inside Google Cloud

Create a JSON file this the following information

Open cloud shell on GCP and type the following.

INPUT_FILE="input.json"gcloud ai-platform predict --model $MODEL_NAME \
--version $VERSION_NAME \
--json-instances $INPUT_FILE

Send a POST request using curl

Note 1: For simplicity I will obtain acces_token using cloud shell using the following command. This will be necessary to send POST request outside GCP.

Note 2: More info

access_token=$(gcloud auth application-default print-access-token)

AI Platform implements a custom predict verb on top of an HTTP POST method.

The endpoint is constructed as follows:



Send a POST request using Postman

Post request as follows:

Returned Status 200 OK and a JSON with the response classes.


Jose Miguel
May 24, 2019

Google Cloud Platform - Community

A collection of technical articles published or curated by Google Cloud Platform Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Jose Miguel Arrieta

Written by

Google Cloud Platform - Community

A collection of technical articles published or curated by Google Cloud Platform Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade