How to Make Predictions Against a SageMaker Endpoint Using TensorFlow Serving

In this article, we will use the endpoint half-plus-three that we created in the previous article How to Create an AWS SageMaker Endpoint for Predictions to make predictions against AWS SageMaker.

AWS CLI and AWS SDK provide sagemaker-runtime , the library responsible to make endpoint requests against SageMaker. The AWS SDK has versions available in the most common computer languages available.

The invoke-endpoint is the operation responsible to make predictions using aws sagemaker-runtime:

using AWS CLI to make requests

#!/usr/bin/env bash 
ENDPOINT_NAME=half-plus-three 
aws sagemaker-runtime invoke-endpoint \
--endpoint-name ${ENDPOINT_NAME} \
--body '{"instances": [1.0,2.0,5.0]}' prediction_response.json 
cat prediction_response.json

using boto3

import json 
import boto3 
client = boto3.client('runtime.sagemaker')
 
data = {"instances": [1.0,2.0,5.0]} 
response = client.invoke_endpoint(EndpointName='half-plus-three',
                                  Body=json.dumps(data))
response_body = response['Body'] 
print(response_body.read())

using node

var AWS = require('aws-sdk');
var sageMakerRuntime = new AWS.SageMakerRuntime();
var params = {
  Body: new Buffer('{"instances": [1.0,2.0,5.0]}'),
  EndpointName: 'half-plus-three'
};
sageMakerRuntime.invokeEndpoint(params, function(err, data) {
  responseData = JSON.parse(Buffer.from(data.Body).toString('utf8'))
  console.log(responseData);
});

You have access to the entire list of examples below.

Related Articles:

How to Create a TensorFlow Serving Container for AWS SageMaker

How to Push a Docker Image to AWS ECS Repository

Did you use the container? Do you have questions or requests for more articles? Add comments below!