What’s new in Watson OpenScale: beta experience with advanced metrics, Fast Path tutorial

Susannah Shattuck
May 7 · 3 min read

With Watson OpenScale, we envision helping users understand the impact that AI has on their business outcomes. The new Watson OpenScale beta experience presents the foundational metrics which are important to be collected in order to find the impact on an AI application’s KPIs. This new experience which can be turned on for instances provisioned in IBM Cloud; it gives you access to new quality monitors, to understand model performance. In addition to accuracy, the beta experience allows you to track a variety of new quality monitors to meet your different needs, including precision, recall, F1-measure, and more. These metrics are chosen based on the type of algorithm that you deploy in your model. We have also introduced the capability to define and monitor custom metrics beyond these out-of-the-box options. All of the metrics can be visualized and monitored on the OpenScale dashboards.

Newly designed Watson OpenScale dashboards are available in our beta experience.

Along with the expansion of performance metrics available through Watson OpenScale, the beta experience also provides a deeper look into the payload analytics associated with your models. We have introduced a chart builder that allows you to create custom visualizations, so you can better understand model predictions and inputs at runtime. This chart builder provides an ability to view the output of the model’s prediction against the features or data ranges that a business considers important. It helps uncover new trends in the data which may prompt the business and data science teams to consider changes to the AI model. E.g. in the credit risk example below, you can see the split in predicted classes for different ranges of the attribute Credit History. You can also see how confident the model is, when predicting for these ranges of Credit History.

View detailed quality metrics and payload analytics to understand how your models are performing.
The chart builder allows you to create custom visualizations of payload analytics that matter most to you.

All of these new monitoring features have been wrapped in an updated UI that brings these data to life with beautiful, simplified charts. With more ways to understand model performance the model’s impact on new business data coming in, you will be able to address issues as they arise more quickly and effectively, especially in changing business situations.

Plus, we’ve made it easier than ever to get started with Watson OpenScale with a new Fast Path tutorial that will automatically connect your instance to a sample model and configure your monitors. In less time that it takes to make a cup of coffee, you will be monitoring a credit risk model and exploring the value that Watson OpenScale can bring your AI governance practice. Simply create a new instance of Watson OpenScale on IBM Cloud and try out the fastpath.

If you are interested in becoming a sponsor user of Watson OpenScale, to provide feedback and help us determine the future direction of this product, please let us know.

Simply click “Run demo” to initiate the fast path configuration.
With just one click, you will have a configured instance of Watson OpenScale, monitoring a sample model.

IBM Watson

AI Platform for the Enterprise

Susannah Shattuck

Written by

Currently @Stanford, focused on intersection of AI, policy, and society. Former PM @IBM, where I built tools to help people understand and trust AI systems.

IBM Watson

AI Platform for the Enterprise

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade