Arena for Python models — an interactive dashboard for XAI exploration of predictive models

Published in

ResponsibleML

3 min readDec 8, 2020

Just released a new version of dalex package for Python (v 0.4) brings easy to use dashboard for exploration and cross-comparison of machine learning models (scikit, tensorflow, and many others).

About Arena

When mentioning an arena, you supposedly imagine ancient gladiators fighting for mighty Caesar’s pleasure. It is quite accurate — models are the gladiators here. And you are the Caesar.

The Arena is a tool to x-ray multiple machine learning models from different environments at the same time. You can set together XAI explanations to compare how models behave. Each chart gives you one more perspective to see the Achilles’ heel of any model.

The video above shows instance level exploration. That means we are trying to understand decisions for a given observation. But it’s only one use case. You can read more about more features at https://arena.drwhy.ai/docs.

Charts

The current version of the backend gives you a set of seven powerful plots. Let’s bring some of them.

Break Down is used to decompose a single prediction into contributions of each variable.

The next chart tests how important the variable is for the performance of the model.

Installation

You need to install or update the dalex package to access new features. Just type in that in your console.

pip install -U dalex

How to use it

Each workflow includes the following steps:

1. Initialize Arena

import dalex as dx
arena = dx.Arena()

2. Add models encapsulated in Explainer object

arena.push_model(dx.Explainer(model, y=y_test, data=X_test))

3. Add observations — to create instance-level explanations (optional)

arena.push_observations(X_test)

4. Add datasets — to create data analysis charts (optional, datasets are not used yet)

arena.push_dataset(train, target="y_column", label="training dataset")

5. Start server or precalculate and upload data

arena.run_server() # starts live server
# or
arena.upload_arena() # generates data and uploads it to Gist
# or
arena.save_arena('data.json') # generates data and saves it

Use case — Warsaw apartments

Assume we have two models for the price per square meter of Warsaw apartments. One is the decision tree and the other is the elastic net. Look at this jupyter notebook for details of creating sklearn models.

Arena bases on Explainer objects. They are core objects of DrWhy.ai universe. Explainers provide a unified interface for models from different packages.

exp_elastic_net = dx.Explainer(model_elastic_net, data=X_test, y=y_test)
exp_decision_tree = dx.Explainer(model_decision_tree, data=X_test, y=y_test)

Having models we can initialize a backend for the dashboard.

# create empty Arena
arena=dx.Arena()# push created explainer
arena.push_model(exp_elastic_net)# push whole test dataset (including target column)
arena.push_observations(test)# run server on port 9294
arena.run_server(port=9294)# we can add next models after running server
arena.push_model(exp_decision_tree)

Just that. Now you can explore XAI charts at https://arena.drwhy.ai/?data=http://127.0.0.1:9294/

It is a live mode — fast and temporary. Uploading data gives you the ability to share your work or easily preserve it.

arena.upload()

After a while, this method will print a publicly available URL to the dashboard. Take a look at mine.

What is next

In the next weeks, plots known from R will reach Python. The rising popularity of fairness sets up our priority. The long term goal is support for multi-class classification models.

If you like this new feature let us know at http://dalex.drwhy.ai/.