Deploying Models with Cortex

Data scientists typically like tools with which they can quickly try different techniques (feature engineering and data cleaning) on the data and apply various machine learning algorithms to build machine learning models. Hence, Data scientists prefer to use tools such as Jupyter notebooks that facilitate such exploration. However, Jupyter notebooks are not suitable for the production environment as they do not scale well, i.e., they are are not efficient in dealing with the increase in user load. Furthermore, in production, we have to deal with other concerns such as machines going down, a high response time of the software components (latency), etc.

Microservices architectural patterns are effective in dealing with the problems mentioned above. In the Microservices pattern, the entire system is divided into small autonomous services that can be managed independently. The Microservices pattern deals with the increasing load by adding more microservices to handle user requests. This feature is called Autoscaling. If a microservice goes down, a new one is created seamlessly, which ensures uninterrupted service to the user.

With so many benefits of microservices pattern, a common practice is to convert a model into a microservice. The idea is to wrap the models as a web service and connect with it using a URL over HTTP. However, it’s tough for a data scientist to deal with the nuances of converting a model into a microservices.

Several tools and frameworks are available that hide the complexities of serving a…