Serving models using Tensorflow Serving and Docker

How to serve machine learning models using Tensorflow Serving and Docker

The tremendous potential with Machine learning can only be realized when models are put in production and users can interact with them directly or indirectly.

What is model serving?

Model serving is the process of exposing of a trained model so that it can be accessed by an endpoint. Endpoint here can mean a direct user or other software.

Prerequisites

Some prerequisites for this process are:

• Python (3.5>) and Tensorflow (2.0 >) installed on your local system
• Docker is installed on your local system

Model serving process

Installing Tensorflow Serving

Since you have Docker properly installed, you can use it to download TF Serving.

In your terminal run the following command below:

docker pull tensorflow/serving

This takes some time, and when done, will download the Tensorflow Serving image from Docker Hub.

If you are running Docker on an instance with GPU, you can install the GPU version as well:

docker pull tensorflow/serving:latest-gpu

Building, training, and…