Using Comet’s Model Registry

In Machine Learning, a “model” could be anything. The following pasta machine could be described as a model. It takes “input”, and a set of “hyperparameters”. Well, at least a couple of hyperparameters: the “cutter” determines whether the “output” is fettuccine or spaghetti, and the speed that you spin the crank determines the thickness of the noodles. Change the hyperparameters and you get different output.

A Pasta Machine Learning Model. Photo by Magic Madzik.

Of course, the internals of a pasta machine don’t change over time, and it doesn’t produce better “output” every time you cook (given appropriate “loss” ratings from your dinner guests). That would be cool! So, the metaphor is not complete, but I do like the idea of Pasta Machine Learning.

The pasta machine shows that the idea of a model can be varied between chefs, er, I mean data scientists.

Thanks to scikit learn, models in Python often have a similar interface to one another (for better or worse). For example, models often have methods like:

model.fit()
model.train()
model.test()

Regardless of what kind of model you are using in your Machine Learning code, Comet ML can handle it. Comet has a sophisticated system for logging, registering, versioning, and deploying machine learning models.

The steps of the Comet model pipeline are:

1. Log an Experiment Model, via Comet’s Python SDK Experiment
2. Register an Experiment Model
3. Track Model Versions of the Registered Models
4. Deploy a Registered Model

We’ll go through each of these steps.

1. Log an Experiment Model

To Comet, a model can be composed of any files or folders in the file system. That is, a model can be any collection of files.

The first step in the model pipeline is to log all of the associated files with a model name via an Experiment.

If all of the files connected to a model are in a single folder (and all of the files there are related to the model), you can simply log the model with a name and the path to the folder. Here, we use the name “MNIST CNN” and all of the model files are in the “../models/run-026/” folder:

from comet_ml import Experiment
experiment = Experiment()
experiment.log_model("MNIST CNN", "../models/run-026/")

After you log a model via an Experiment, you will see the model and all of its individual files in the Experiment’s Assets Tab.

The Experiment Model in the Comet.ml UI after running the above code.

From the Experiment’s Assets tab, you can download the model, see if it has been registered before, or register it as a new model, or a new version of an existing model.

Clicking the + Register link will take you to the Registry view:

2. Register a Model

After you log a model through an Experiment through the SDK, you can then register it. Registered Models belong to a workspace, and are shared with your team there.

You can register an Experiment’s model in two ways:

1. Through the Experiment’s Asset Tab in the Comet.ml User Interface
2. Programmatically through the Comet Python SDK

To register an experiment’s model via the Comet User Interface, simply go the Experiment’s Asset tab, locate the model in the asset list, and click on the Register link.

If you have previously registered a model, you can save this model to an Existing version:

However, if you would like to save this as a new model, simply click the “Register New Model” link. That will bring up the Register Model window:

The Register Model dialog.

Here, you can give the registered model a name, a version, and set its visibility to Public or Private.

You can also register an experiment’s model via the Python API. That would look similar to:

from comet_ml import API
api = API()
experiment = api.get("workspace/project/experiment")
experiment.register_model("MNIST CNN")

3. Track Model Versions

Registered models belong to the workspace. In this manner, you can share your models with the workspace team. At the workspace view, you can select between viewing the Projects or the Registered Models:

The Card view of the Registered Models. You can also see the Registered Models in a List view.

Selecting a registered model’s card in the Model Registry will take you to the detailed view of the registered model:

The Detailed View of the Registered Model.

Here you can see the latest version of the registered model, and all previous versions as well. Registered models contain the following properties, which can all be edited here:

• Registered name
• Description
• Visibility setting (public vs. private)
• Notes

4. Deploy Models

To download a registered model as a zip file, click on the “Get Model” button. The zip file will contain all of the files that were logged with the Experiment model via the Python SDK.

The Versions of the Registered Model.

You can download the latest versioned registered model, or any of the versions. Simply click on their download icon.

That’s it! I hope you enjoyed our quick snack overview of Comet’s Model Registry!

For more information, please see: comet.ml/docs/user-interface/models/

Bon appétit!

Originally published at https://www.comet.ml on September 9, 2020.