H2O MLOps Beginner’s View

When we step into the AI application world it is not one easy step. It has a series of tasks that are combined. To convert an idea to the workable stage we must fulfill the requirements in each stage. When we seek existing platforms, there are leading solutions in the industry to fulfill each stage of the pipeline. The specialty of the H2O platform in my view is that it has the solution to each step in the pipeline.

Make - Build models and applications with accuracy. (Driverless AI and H2O Wave)

Operate - Machine Learning Operations to monitor and rapidly adapt to changing conditions. (H2O MLOps)

Innovate - Deliver innovative solutions to the end-user. (H2O AI App Store)

Figure 1: H2O End-to-End Process

Here I am going to focus on Operate section which mainly works with MLOps.

What is MLOps?

MLOps is the short-term of Machine Learning Operations. It is kind of a process that we can define as a seamless handoff between the Data Scientist and ML Engineer or the developer who takes the models to production. When we figure out the big picture of MLOps we can identify the main three goals.

1. Faster experimentation and model development.
2. Faster deployment of updated models into production.
3. Quality Assurance

Main principles in MLOps

1. Automation — Automate the workflow steps without any manual intervention.
2. Versioning — Track ML models and data sets with version control systems.
3. Experiment Tracking — Multiple experiments on model training are parallelly executed.
4. Monitoring — Monitor to assure that the ML model performs as expected.
5. Testing — Tests for features and data, model development, and infrastructure.

How does it differ from DevOps?

The application of DevOps principles and practices to the Machine learning workflow can be identified as MLOps. Mainly when we talk about the usage of these two methods DevOps comes up with software development of web development projects. Those kinds of implementations mainly follow up CI/CD approach as a pipeline. In MLOps in addition to the CI/CD, there is an additional stage in the pipeline called as Retrain approach. This is because MLOps mainly relate to the Machine learning projects in which the development life cycle is a bit different from the other application developments as well as the ML or AI solutions have a chance to be outdated.

Mainly these two categories create different job roles for the team members. In DevOps, Software Engineers develop the code itself while DevOps Engineers are focused on deployment and creating a CI/CD pipeline. When it comes to MLOPs Data Scientists play the role of Software Engineers as they write codes to build models while MLOPs Engineers are responsible for the deployment and monitoring of these models in production.

Why H2O MLOps?

In H2O MLOps I could find three main sections which are called Model Repository, Model Deployment, and Model Monitoring. The cool nature of their platform is the flexible architecture that is more than supportive to perform ML Operations at the production level. We can find out three main pillars to describe the flexibility of the platform.

1. Extensible — H2O AI Cloud platform has clients for python, R, and Java which benefits users with the latest versions of open-source packages. In MLOPs this will be helpful for users to train, deploy and customize both H2O.ai as well as third-party models.
2. Distributed — Due to the Kubernetes-based deployment approach in the platform, resource allocations are handled automatically. In MLOPs this will be a great benefit as we can deploy and handle any data size and model training that occurs across multiple CPUs and GPUs.
3. Scalable — Using existing high-performance NVIDIA GPUs and CUDA runtime to cater to increase of requirements. In MLOPs this will be a great supportive feature in the platform to provide high-performance noninterrupted service to the user.

With the above-mentioned flexible AI platform, we can dig into the main sections covered in the H2O MLOPs.

1. Model Repository

This is a kind of management section in the H2O MLOps. This means their platform act as a central place in the organization which can host and manage all experiments and their associated artifacts. Here we can register our experiments as models and maintain transparently by using versioning. Other than models trained from Driverless AI we can use any other third-party model in the H2O MLOps platform.

2. Model Deployment

Here we can build a model once and deploy it in different methods. Common deployment modes are known as multi-variant (A/B) mode and champion/challenger mode. As an example, in the above-mentioned modes, we can automatically filter out which version of the deployed models will perform well in the production and use it.

3. Model Monitoring

This section in the H2O MLOps maintains model oversight. Other than basic software development, in the ML project, the solution we provided for a problem may drift with time. This is because of the change of data and concept. Other than that, in H2O MLOps they provide model feature importance results, experiment leaderboard, and receive alerts and notifications.

The practicality of End-to-End Solution.

Here in the below steps, I have illustrated a few steps I tried out by using the H2O MLOps Platform. It is quite easy and user-friendly to work with their amazing and simple UI.

1. Create a project- Once logged in to the H2O MLOps you can see an interface as below and you can create your project.

Figure 2: H2O MLOps Platform

Figure 3: Create New Project

Figure 4: After Creating New Project

2. In Driverless AI now we can see the project we created, and we can link our experiment we ran in DAI to that project. Here I ran an experiment related to the Melbourne House Price Data set and after the link, we can see our model is added to the MLOps platform.

Figure 5: Driverless AI Updated Project List

Figure 6: Create Experiment in DAI

Figure 7: Inside view of the Empty Project

Figure 8: Link Experiment to the Project

Figure 9: MLOps Platform Updated with Linked Experiment

3. Other than directly linking the Driverless AI model to MLOps we can import our models manually to the MLOps platform. To check that I ran another experiment to the same dataset and downloaded Mojo Scoring Pipeline. Then I imported it to the MLOps platform.

Figure 10: Download MOJO Pipeline from DAI Experiment

Figure 11: MOJO Download Page

Figure 12: Add Model Interface in MLOps Platform

Figure 13: Drag and Drop Downloaded MOJO from DAI

4. We can deploy our imported model in the Dev (Development) platform or Prod (Production) platform. Initially, I tested the Single model deployment which is the simplest one out of three. After the deployment, we can see it in the deployments section with some useful data. In the action bar, we have lot more options as shown below.

Figure 14: Model Action Bar

Figure 15: After Deploying Model

Figure 16: Action Bar in Deployments

Figure 17: More Details about the Deployment

Figure 18: “Show Sample Request” Action

Figure 19: Sample Request Output

5. As the next deployment method, I tried out the A/B Test deployment. Here we have to select two or more models and click the “A/B Test”. This will compare the performance of selected models and at the early stage, we have to specify the percentage of traffic each model should have to receive.

Figure 20: Select Multiple Models to the A/B Deployment

Figure 21: Add Traffic Percentages

Figure 22: View After the Deployment

6. As the third and final deployment method, I tested Champion/Challenger deployment. Here the MLOps let us compare our chosen best model (Champion) to a challenger model continuously. We have to select the action bar in front of the challenger model and fill the popup menu and deploy it as below.

Figure 23: Challenger Selection Action

Figure 24: Champion Model Selection Step

Figure 25: Deployment Environment Selection Step

Figure 26: Add Security Level

Figure 27: After Deploying in All Three Formats

7. After the deployment, we can continuously monitor the performance of the model including alerts and warnings. This will open in a Grafana dashboard. I have added sample output collected from the H2O MLOps documentation.

Figure 28: Model Monitoring Step

Figure 29: Model Monitoring Step

In the end, what I noted was H2O MLOPs is still in the release version 0.52.1 which provides amazing MLOps services. Hope it will be a fascinating product when it reaches release 1.0.

Rather than using different platforms when working on an ML project, it is better to have a single platform that helps to convert the idea and data to a usable end product. I Hope H2O.ai fulfills this requirement with their products.

If you are a beginner TRY this. Surely you will absorb more knowledge with hands-on experience.