How to Train and Deploy a Machine Learning Model Using LifeOmic Patient ML Service
TLDR: You can use LifeOmic’s Patient ML Service to easily deploy your existing supervised machine learning (ML) model code. This will let you quickly get your model to production and begin to realize value from it. MLOps best practices, HIPAA compliance, and security are all included.
Patient ML Service
Training, deploying, managing, and versioning machine learning (ML) models is hard. And doing it on patient or medical device data in a HIPAA-compatible way is even harder. That’s why we’ve created the LifeOmic Patient ML Service. We’ve created a platform that lets you easily deploy a supervised ML model into a production environment without having to write any complex ML infrastructure code related to training dataset curation, data labeling UI integration, hyperparameter optimization, model evaluation, model versioning, model deployment, or model monitoring. All of that is included in the service.
In this tutorial article we’ll go through how to use your LifeOmic account to easily bring your ML model online into a robust, production-ready state, set up for continuous training and evaluation so your model can ride the data drift and not become stale. This tutorial assumes you already have training and inference code for your ML model and already have your training data living as FHIR records in your LifeOmic account.
Note: Our Jupyter notebooks service is a perfect place to securely experiment with your data and write your initial model code. Once you have code for a model you’re happy with, bring it to Patient ML Service to make productionizing your model easy.
Core Concepts
Patient ML Service has the idea of model configs and model runs. A model config describes things about your model, including what its problem type is, what labels it uses, how to source its training data, its hyperparameter optimization space, and a pointer to a Docker image containing your model code. A model run represents a training run and trained version of your model. A model run has artifacts associated with it such as an immutable, model-consumable training dataset snapshot of your data (an input to the model run), training, validation, and test set metrics, and a trained model artifact (outputs of the model run).
Our Patient ML Service API (docs here) is public and provides a comprehensive way of interacting with the service.
Our service includes many best practices baked in. Some examples include automated re-training, hold-out test set evaluation, and time-based cross validation for more accurate in-the-wild performance estimation. Also, we impose no restrictions on what runtime, language, or dependencies your model has to use.
Defining Your Model
The first step is to create your model config. We do that below by calling the Create Model endpoint of the Patient ML Service API. The code below expects them to be available in environment variables.
Note: To authenticate with LifeOmic, you’ll need to pass in headers containing your LifeOmic Account ID and API key. For more information, see our Getting Started Guide.
import os
headers = {
"LifeOmic-Account": os.environ["LIFEOMIC_ACCOUNT"],
"Authorization": f"Bearer {os.environ['LIFEOMIC_API_KEY']}",
}import requests
res = requests.post(
"https://api.us.lifeomic.com/v1/patient-ml/models",
json={
"name": "EBHI-SEG Demo",
"description": "A multi-class semantic segmentation model trained on the EBHI-SEG dataset to detect colorectal cancer in enteroscope biopsy images.",
"problemDefinition": {
"problemType": "imgSeg",
"trainingDataFilter": {
"filterType": "FhirCodesFilter",
"codes": ["28019-8"],
},
"labelDefinition": {
"labels": [
{
"name": "Adenocarcinoma",
"description": "",
},
{
"name": "High-grade IN",
"description": "Indicates the presence of high-grade intraepithelial neoplasia",
},
{
"name": "Low-grade IN",
"description": "Indicated the presence of low-grade intraepithelial neoplasia",
},
{
"name": "Normal",
"description": "Indicates normal, healthy tissue.",
},
{
"name": "Polyp",
"description": "",
},
{
"name": "Serrated adenoma",
"description": "",
},
],
},
},
"trainingApproach": {
"type": "tuningJob",
"trainingImage": "<your_ecr_image_uri>",
"maxTrials": 10,
"searchSpace": [
{
"name": "epochs",
"scale": "linear",
"min": 1,
"type": "integer",
"max": 50,
},
{
"name": "lr",
"scale": "log",
"min": 0.00001,
"type": "continuous",
"max": 0.01,
},
{
"name": "batch_size",
"type": "categorical",
"values": ["2", "4", "8", "16", "32", "64"],
},
{
"name": "img_size",
"type": "categorical",
"values": ["128", "256", "384"],
},
],
"metricDefinitions": [
{"name": "val_dataset_iou", "regex": "val_dataset_iou=([0-9\\.]+)"},
{"name": "loss", "regex": "loss=([0-9\\.]+)"},
{"name": "train_dataset_iou", "regex": "train_dataset_iou=([0-9\\.]+)"},
],
"objective": {"metric": "val_dataset_iou", "direction": "maximize"},
},
"deployApproach": {
"type": "edge",
"inferenceImage": "<your_ecr_image_uri>",
},
},
headers=headers,
)
print(res.status_code, res.text)
model_id: str = res.json()["model"]["id"]There’s a lot going on in that model config. Let’s dissect each part so we can understand:
Problem Definition
problemDefinition is how we tell Patient ML Service what specific machine learning problem we’re trying to solve. There are a few subfields:
problemType— This specifies the supervised machine learning problem type for our model. We currently support semantic image segmentation and image classification (binary, multi-class, and multi-label). We are excited to support additional problem types soon!trainingDataFilter— This is how we tell Patient ML Service what training data to curate for our ML model from our LifeOmic account. We can specify FHIR system codes, and the service will pull in any FHIR records that have a system code (type.coding.code) matching any of the codes in our filter. In this example, we specify28019–8which is the LOINC code for “Enteroscopy Study observation”.labelDefinition— This is where we specify which labels our model knows how to predict. Once the config is created, the service assigns a unique ID and unique integer index to each one.
Training Approach
trainingApproach is where we tell the service how to train our model. The subfields are:
type— This must betuningJob, and means the service will conduct automatic hyperparameter optimization on our model, by training multiple candidate model versions in each model run. The number of candidate versions in each run is controlled by themaxTrialsfield.trainingImage— An AWS ECR image URI for a Docker image that contains your model training code.searchSpace— This defines the hyperparameter search space of your model that will be explored during each model run. The search finds the model version that best generalizes to the validation dataset the service creates for you. In this example, we are providing four model hyperparameters to tune.metricDefinitions— This defines the metrics the service will extract from the logs produced by your model over the life of its training. Once the best model version is found during a run, the final value for each of your metrics will be saved with the model run and viewable in the LifeOmic Platform UI.objective— This specifies the metric frommetricDefinitionsthat the service should optimize during hyperparameter optimization.
Deploy Approach
deployApproach is where we tell the service how our model should be deployed. There are just a couple subfields:
type— One ofcloudoredge. In thecloudcase, the service will automatically deploy for you any version of your model that you approve. It will also allow you to query it for predictions via the service API. In theedgecase, the service will not deploy the model, but will still keep track of the approved version. This allows you to download the model artifact yourself and run it on your own hardware.inferenceImage— An AWS ECR image URI for a Docker image that contains your model inference code. Even iftypeisedge, this value must be supplied so the service can evaluate your model on a hold-out test set.
Training Your Model
Once your model config is created, our service has everything it needs to train your model. Training is as easy as sending a body-less POST request to the service API’s Create Run endpoint:
res = requests.post(
f"https://api.us.lifeomic.com/v1/patient-ml/models/{model_id}/runs", headers=headers
)
print(res.status_code, res.text)
run_id: str = res.json()["runId"]When a run is created, its parent model config is used to determine what training data to curate, what model to train, how to optimize its hyperparameters, and how to evaluate it. Our service handles all the ETL of curating your production FHIR data into immutable training/validation/test dataset splits. We automatically copy the training and validation splits to your training image when we run it. After the best model version is found during hyperparameter optimization, we evaluate it on the hold-out test dataset. That new candidate model version is called the challenger. We also evaluate the current approved (usually the overall best) version of your model that you have running in production (the champion) on that test set as well. In this way, after the run is complete, you have an apples-to-apples comparison of how the new model version compares against your current best model version. This empowers you to make the objective decision of whether to promote your new model version to production and make it the new champion.
There are a few ways you can view the progress and results of your model runs. You can query the service’s Get Run or Get Runs endpoints directly to inspect programmatically, or you can view them in the Machine Learning tab of the LifeOmic Platform UI using our Runs Table and Metrics views (pictured below).
Deploying Your Model
If you decide your new model version is better than your current champion, you can choose to deploy it. This makes it the new champion version that will serve production prediction requests. Doing so is as easy as creating an “approval” for your model version:
res = requests.post(
f"https://api.us.lifeomic.com/v1/patient-ml/models/{model_id}/runs/{run_id}/approvals",
json={"choice": "approved"},
headers=headers,
)This model version is now the model’s official champion version. For cloud type models, the model version’s model artifact will be automatically deployed and replace the old champion with zero downtime.
Since the model in this demo is the edge type, we can use the Get Model Artifact service endpoint to download the model artifact for our current champion, then load it into our runtime and run prediction requests through it.
Wrapping Up
This service abstracts away of lot of the complexity of running machine learning models in production, while simultaneously providing you with MLOps best practices, HIPAA compliance, and security. If you’re looking to get started, we’d be more than happy to help! Please reach out on LinkedIn or you can email us at info@lifeomic.com.
