The 5 Components Towards Building Production-Ready Machine Learning Systems

Deploying Models Into Production Is A Complex Affair

Introduction

The biggest issue facing machine learning is how to put the system into production. Machine learning systems differ from traditional software in two fundamental ways:

• Machine learning is never fully deterministic; therefore, the performance of an ML system can’t be evaluated against a strict specification. Instead, it should always be evaluated against application-specific metrics (false positives/negatives, churn rates, sales)
• The behavior of a machine learning system is determined more by the data used for training than the model used for inference. Therefore, data collection, data wrangling, pipeline management, model retraining, and model deployment are tasks that will never go away.

Sergey Kareyev at Full Stack Deep Learning Bootcamp November 2019 (https://fullstackdeeplearning.com/november2019/)

The diagram above displays a conceptual framework for the different components of a Machine Learning System:

• Prediction System: This is the system that involves code to process input data, to construct networks with trained weights, and to make predictions.
• Training System: This is the system with the code to process raw data, run experiments, and manage results.