Deploying Azure Machine Learning Containers

Guide to a few ways to publish ML models to Azure

So many ways to do it in Azure Land

Azure has several offerings in Machine Learning space and it is evolving quickly. It can be confusing to navigate while the ground is still moving:

• Azure ML Workspace — combines Jupyter Notebooks, Azure’s own AutoML, Compute resources, Model inventory, etc — along w/ a cheeky Visual ML builder:

Other than being in Preview/Beta, this isn’t a bad bundle of tools (and how much does it cost — I think it is free — I hope ?).

• FYI below is the older version of this.. again Azure is moving fast!

Training the Model with Azure Notebooks

Using Azure ML WS — you can leverage Jupyter notebooks online, though I wonder why run it on the cloud… anyways I used my basic diamond dataset and trained a basic ISO Regression model:

Trained the most basic of models, output is a pickle file (serialized trained model)

Deploying the Model into Azure ML Containers

Assuming you have an Azure account, you need to first create a Machine Learning Workspace (DiamondMLWS in the below example).

Basic 3 steps:

1. Register your models (your pickle [mymodel.pkl] files)
2. Env file setup & Scoring callback file (.py interface)
3. WebService (AciWebService), Model, InferenceConfig, and deploy.

Registering is really easy, do it once per model (pkl file):

ws = Workspace.get(name='DiamondMLWS',
    subscription_id='****',resource_group='diamond-ml')new_model = Model.register(model_path="mymodel.pkl"model_name=name,
    description="mymodel descr",workspace=ws)

Env file can be generated w/ a script like this (one time):

myenv = CondaDependencies.create(conda_packages=['numpy',
   'scikit-learn'])
with open("myenv.yml","w") as f:
   f.write(myenv.serialize_to_string())

Scoring file looks like this implementing init() and run() for each model:

#score_mymodel.py
def init():
   global model
   model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'),
        'mymodel.pkl')
   model = joblib.load(model_path)def run(raw_data):
   data = np.array(json.loads(raw_data)['data'])
   y_hat = model.predict(data)
   return y_hat.tolist()

The final step involves setting up the Service, Model, and deploying once per model:

aciconfig = AciWebservice.deploy_configuration(cpu_cores=1,
   memory_gb=1,tags={"data": "diamonds","method": "sklearn"},
   description='Predict Diamonds with sklearn')
ws = Workspace.get(name='DiamondMLWS', subscription_id='****',
   resource_group='diamond-ml')
model = Model(ws, name)inference_config = InferenceConfig(runtime= "python",
   entry_script="score_mymodel.py", conda_file="myenv.yml")service = Model.deploy(workspace=ws,
   name=svcname, models=[model], inference_config=inference_config,
   deployment_config=aciconfig)
service.wait_for_deployment(show_output=True)

That is it! You should see output declaring that the service has been deployed and returns the URI to access, and check on the Azure portal to see it is there. Also keep an eye on costs — more on that in the end :)

Running.......................
ACI service creation operation finished, operation "Succeeded"
http://dds2f-97f5eb5c4163.eastus.azurecontainer.io/score

That is pretty much it. Input and output is what the model inputs/outputs are from the Jupyter code. You can test the API with Postman or curl — make sure it is pricing:

Priced 4 diamonds — 0.5 to 2.1 carats — see the predicts! $1.3k to $25k !

View of (older) Azure ML Studio

• This has been replaced by the Visual Interface that is part of the latest ML Workspace. However this old one is still free — the newer Visual Interface is not !

Fancy looking but not entirely operational

I used Azure Studio to build a few models — Linear Regressor, Boosted Tree, and a Neural Network to price diamonds given the input 4C’s (carat, color, clarity, cut !!). It is pretty good for folks who can’t type. Main takeaways here:

• The GUI is great for management demo’s, but it is overly basic and does not provide a very wide suite of functions.
• Param and Hyper-Param tuning of provided models is very very limited.
• You are able to quickly publish to a free webservice URL with few clicks which is very nice.
• The entire thing is free, you don’t have to pay for anything!
• Overall I would never use this in prod.. but it looks nice for demos when you want to ask for funding...

Plan B — Building your own web container

Another approach is to just have an inference engine in Python tacked onto an existing Python webserver (Django or Flask). This is maybe alot more simple than the Azure ML Container approach:

app = Flask(__name__)
models = {}
models['ISO'] = joblib.load('mymodel.pkl')
models['LR3'] = joblib.load('mymodel2.pkl')
@app.route('/models/<model>', methods=['POST'])
def predict(model):
   if (models.get(model) is None):
      return jsonify("[-1]")
   j_data = np.array(request.get_json()['data'])
   y_hat = np.array2string(models[model].predict(j_data))
   return y_hat
if __name__ == '__main__':
   app.run(debug=True)

The Flask code above deployed to its own Web App using Kudu costs about $14/month (@ B1 cheapest Dev/Test HW). This container can host any number of inference models as seen above.

The reason for Plan B is that ML Azure Containers are expensive!! $2/day per model since each is deployed it its own container (or $60 month * # models). I ran up a huge bill and quickly retired it in favor of the Flask approach.

I am sure this article will be out of date in no time.. Azure ML is moving quickly. Look for more posts in this domain!

References:

Foo, Doug. “Full Stack Machine Learning on Azure” Medium.com. 29 Nov. 2019. (Website)