Introducing Myelin

The main intention behind Myelin is to build a tool that makes it easy to deploy Machine Learning (ML) models on Kubernetes. Since I started working as a data scientist, the biggest challenge I found was how to bridge the gap between data science and engineering, so that algorithms can be built and rolled out to users without too much pain.

On Kubernetes applications can be containerised, so that any code written by data scientists can be taken as is and executed on the platform.

Kubernetes is a declarative system, which is well suited for ML deployments, since it enables users to think about the state they would like to achieve, not the steps to get there. For example an algorithm deployed behind an API and computing/processing predictions for each request (with reasonable accuracy and within certain response time) is a state that a ML deployment could aim for.

Kubernetes also allowed us at Myelin to integrate with many different tools and technologies, so that we can provide a tool that is flexible and easily extendable.

Main components

The main building blocks of Myelin are sensors and tasks, defined in a yaml file that is called an axon. Tasks are executable steps, whereas sensors are event driven triggers that can trigger some tasks.

Here is a basic example. The user can define the following blocks:

Tasks:

• Pre-processing task: Download some training data
• Training task: Train and evaluate a ML model
• Deploy task: Wrap the model in a REST API

Sensors:

• Trigger sensor: trigger some tasks for the first time
• Post training sensor: check the accuracy of the trained model and execute the deployment steps
• Post deployment sensor: monitor the accuracy of the deployed model and retrain it if it drops below a certain threshold

Once these building blocks are designed they can be connected as follows:

• The trigger sensor executes the pre-processing and the training task.
• The post training sensor listens to a train complete event (and whether the model is accurate) and executes the deployment task.
• The post deployment sensor listens to an event that is triggered when the accuracy of the deployment is below a certain threshold and executes the training and pre-processing steps, closing the loop in this case.

In addition to the execution model there are more features, a few worth highlighting:

• Hyperparameter tuning: Myelin has a built in hyperparameter tuning that can be executed as part of the training step, using Hyperband (https://arxiv.org/abs/1603.06560) under the hood.
• Deployment routing strategies: Users can define complex routing strategies at multiple levels for each deployment, using Istio components.
• Deployment execution DAG: Multiple models can be deployed next to each other in a flexible DAG.

If you would like to try Myelin, installation steps are posted on our website for AWS, Azure and GCP (https://myelin.io/#blog).