Deploying a machine learning model in Azure ML

How to create a callable endpoint using a registered Azure ML mlflow model and integrate it in a web app.

5 min readSep 28, 2023

Introduction

In the previous article: Creating a MLOPS pipeline with Azure Machine Learning Service | by Martha | Sep, 2023 | Medium we created the code base and pipeline to train a simple machine learning model using mlflow in Azure ML. At the end we had a registered model in Azure ML that we will now deploy to an endpoint. We will be looking at the code needed for step 4 of the MLOps lifecycle.

Step 4: Deploy

In the deploy step we expose the model to other users so that they can make requests and receive predictions. The deployment is happening in Azure ML and we will test the endpoint locally.

We will be creating an endpoint in Azure ML.
We will deploy our registered model to this endpoint.
We will create a streamlit app locally.
We will integrate the endpoint with the app.
We will call the endpoint to get a prediction.

The create_or_update_endpoint job:

jobs/create_or_update_endpoint.yml

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: ENDPOINT_NAME
auth_mode: aml_token

The endpoint is created using the az cli in the following order:

create
deploy the model to the endpoint (see below)
update to define how much traffic is routed to it
invoke it using a request file (.json see below)

az ml online-endpoint create -n 'endpoint_name' -f ./jobs/create_or_update_endpoint.yml -g "rg" -w "ML-Workspace"

az ml online-endpoint update -n 'endpoint_name' --traffic "deployment_name=100" -g "rg" -w "ML-Workspace"

az ml online-endpoint invoke -n 'endpoint_name' --request-file sample_request.json -g "rg" -w "ML-Workspace"

The deploy_model_to_endpoint job:

jobs/deploy_model_to_endpoint.yml

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: deployment_name
endpoint_name: endpoint_name
model: azureml:model_name:version
instance_type: Standard_DS3_v2
instance_count: 1

az ml online-deployment create -f jobs/deploy_model_to_endpoint.yml -g "rg" -w "ML-Workspace"

The deployment itself can take a few minutes. Once it’s done you can directly test it in the UI.

Update and also invoke the endpoint with the last two commands from the previous step.

The endpoint setting should change to:

Invoke should return:

The streamlit app:

Before running the streamlit app we need to update our config.yml with the endpoint information (fill out all the x’s):

connections:
  tenant_id: "xxx"
  subscription_id: "xxxxx"
  resource_group: "xxxxx"
  workspace: "xxxxx"
deployments:
  job_name: ""
  endpoint_name: "xxx"
  endpoint_url: "https://xxx.ml.azure.com/score"
data:
  sample_request: ""
  file_name: ""
  datastore: ""
  train_data: "train.parquet"
  test_data: "test.parquet"
model:
  registered_name: "credit_defaults_model"
  version: 1
  workspace_id: ""
  folder_path: "model"
training:
  job_name: ""
  environment: ""
  experiment: ""
  test_train_ratio: 0.0
  learning_rate: 0.0
compute:
  cluster_name: ""
  image: ""
  environment_name: ""
  conda_file: ""

Update your requirements.txt and install additional libraries:

azureml-core==1.51.0
azureml-defaults==1.51.0
azureml-mlflow==1.51.0
azureml-telemetry==1.51.0
azure-ai-ml==1.9.0
azure-identity==1.14.0
debugpy==1.6.3
fsspec==2022.1.0
ipykernel==6.0
matplotlib==3.7.2
numpy==1.21.5
pandas==2.0.3
pre-commit==2.15.0
psutil==5.9.5
PyYAML==6.0.1
scikit-learn==1.3.0
scipy==1.7.3
streamlit==1.27.0
tqdm==4.65.0

Now we can create a simple app to call the endpoint (streamlit.py):

import json

import requests
import streamlit as st
import yaml
import json

from mlops.utils import ml_connect

# Read configuartion 
cfg = []
with open("config/config.yml", "r") as ymlfile:
    cfg = yaml.safe_load(ymlfile)

# Connect to Azure ML Workspace
ml_client = ml_connect(credential_type="default", cfg=cfg)

# Get endpoint acces key
endpoint_cred = ml_client.online_endpoints.get_keys(
    name=cfg["deployments"]["endpoint_name"]
).access_token

# Define request header
headers = {
    "Authorization": f"Bearer {endpoint_cred}",
    "Content-type": "application/json",
}

# Define the request
request_json = []
 
with open("data/request.json", "r") as f:
  request_json = json.load(f)

st.title("Calling Azure ML endpoint.")

endpoint = cfg["deployments"]["endpoint_url"]

# Call endpoint
if st.button("Call Endpoint!"):
    res = requests.post(url=endpoint, data=json.dumps(request_json), headers=headers)

    st.subheader(f"status code: {res.status_code}, response: {res.text}")

utils.py

"""Module to connect to ML workspace."""
from mlflow.tracking import MlflowClient
from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential
from azureml.core.authentication import InteractiveLoginAuthentication
import mlflow
import yaml


def load_config(config_path: str) -> dict:
    """Function to load config file."""
    cfg = []
    with open(config_path, "r") as ymlfile:
        cfg = yaml.safe_load(ymlfile)
    return cfg


def mlflow_connect(credential_type: str, cfg: dict):
    """Function to connect to MLflow."""
    ml_client = ml_connect(credential_type, cfg)

    mlflow_tracking_uri = ml_client.workspaces.get(
        ml_client.workspace_name
    ).mlflow_tracking_uri

    mlflow.set_tracking_uri(mlflow_tracking_uri)

    return MlflowClient()


def ml_connect(credential_type: str, cfg: dict) -> MLClient:
    """Function to connect to ML workspace.

    Args:
        cfg: Dict with config values.
        credential_type: Type of credential.

    Returns:
        MLClient.
    """
    credential = get_credential(credential_type)

    try:
        credential.get_token("https://management.azure.com/.default")
    except Exception:
        credential = InteractiveLoginAuthentication(
            tenant_id=cfg["connections"]["tenant_id"]
        )

    return MLClient(
        credential,
        cfg["connections"]["subscription_id"],
        cfg["connections"]["resource_group"],
        cfg["connections"]["workspace"],
    )


def get_credential(credential_type: str) -> DefaultAzureCredential:
    """Function to get credential.

    Args:
        credential_type: Type of credential.

    Returns:
        Credential.
    """
    if credential_type == "default":
        return DefaultAzureCredential()
    elif credential_type == "interactive":
        return InteractiveBrowserCredential()
    else:
        raise ValueError("Invalid credential type.")

The sample request:

{
    "input_data": {
      "columns": [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22],
      "index": [0, 1],
      "data": [
              [20000,2,2,1,24,2,2,-1,-1,-2,-2,3913,3102,689,0,0,0,0,689,0,0,0,0],
              [10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 10, 9, 8]
          ]
    }
}

The result looks something like this:

streamlit run streamlit.py

Summary

In this tutorial we went through the fourth step of the MLOPS lifecycle using Azure ML and streamlit. We have created an endpoint in Azure ML, deployed our previously registered model to it and update the endpoint to accept all traffic. In the last step we embedded a link to the endpoint in a local streamlit app and called the model from a website running on localhost.

If you found this article useful, please follow me.