MLOps in a nutshell

Credit — https://www.slideshare.net/weaveworks/using-mlops-to-bring-ml-to-productionthe-promise-of-mlops

Good day! you might have heard about ‘ML Ops’ being used recently more often in the field of Data Science. It is the set of operations involved setting up of infrastructure; deployment & maintenance of Machine learning algorithms in the production environment. This is particularly interesting because there are a lot of Data Scientists who work on complex algorithms; struggle to achieve the highest accuracy, however, it is not going to generate profit for the company unless it is put to work.

There are a lot of ways to deploy an ML algorithm. Few are listed below,

API Programming Language —

Python - Write a Flask app (interacting with the ML model) which acts as an API Layer to serve the model. The flask app needs to use gunicorn library to serve HTTP requests in production.

The API layer (along with the model) can be deployed in the below environments —

1. The API layer can be deployed in Docker. This is universally adopted across many organizations because it is platform-independent. ie) If the company decides to move from one cloud provider to another (For Instance — AWS to Google Cloud or Microsoft Azure), it is easier to just deploy the docker containers and it should work perfectly fine.

Realtime systems (particularly those ones which has strict response timelines such as less than 100milliseconds) would expect faster processing to avoid delay. This brings us to the need for ‘Load Balancing’.

Load Balancing option allows the systems to handle heavy traffic by diverting them to slave process so that one system isn’t overloaded by incoming requests. Either ‘NGINX’ + ‘Docker Swarm’ (or) Kubernetes can be used for load balancing.

Instructions on how to deploy container application in GKE (Google Kubernetes Engine) is given here. AWS Cloud also provides a similar service called ECS (Elastic Container Service).

2. Easier alternative (but works only in AWS) is to use ‘AWS Elastic Beanstalk’. An article on how to deploy a Python Flask application is given here. The platform (though its a bit expensive compared to Docker) handles the load balancing on its own and therefore, you can sit back and relax !. It is strongly recommended for smaller companies who can’t afford to hire a DevOps resource to manage the cloud infrastructure.

After completing (1) or (2), you will have a static IP, which can now be tested with tools such as ‘Postman’ (for doing some basic API calls) & JMeter (or) Postman for performance/load testing.

Note: If you’re using Docker, you may also test the API performance locally before deploying them in GKE or AWS ECS. However, it may not give you the real numbers on the load test performance ie) exact performance of the API in production. A practice (seen in few companies) is to deploy the API in a new Kubernetes cluster in Development Environment. After performing the load tests, it is again deployed in Kubernetes Cluster on the Production Environment.

Feel free to reach me on LinkedIn. You may read other articles under my publication

👏 if you liked my post! Have a fabulous day! 😃