How to EASILY put Machine Learning Models into Production using Tensorflow Serving

MACHINE LEARNING is fun, right?? Well.. most of the time, but putting models in production sometimes sucks especially in a rapidly growing field like Machine Learning. How do you keep track of your models? How do you implement source control on your beloved models? How do you serve them quick enough for real life cases?These are some of the issues I am running into on cool top secret project at Walmart Labs, Finding good examples and documentation on this topic is challenging, so I thought I would share my findings.

Tensorflow Serving is the obvious and best solution to bring machine learning models to production. The benefits include:

1. A pre-built rest api that is super fast and backed by google
2. Source control and access easy to different models and version of those models
3. A Standardized way to serve models across all platforms

However, because it is new there are a lot of things missing like DOCUMENTATION(seriously google??) and documented support for other popular libraries like Keras. Keras is fully compatible with Tensorflow Serving the only challenge is saving the models in such a way that Tensorflow Serving can work with it. Thus, this tutorial series is an attempt to get one quickly up and running with a production API that is easy to use and solves many issues one might run into when trying to do this.

I will break this up into 3 Main parts, click any of parts below to be directed to the proper Article.

1. Setting Up and Configuring Tensorflow Serving
2. Training and Saving a Prediction Model for Tensorflow serving
3. Training and Saving a Classification Model for Tensorflow serving

Let me know if I should add anything or be more descriptive.

— — Brian Alois Schardt