How to Access Data in an Evidently AI Report

Evidently AI is a powerful tool for monitoring and analyzing the performance of ML models. This article will guide you through the process of generating a report using Evidently AI, saving it in JSON format, and then accessing specific metrics from the report. We will use a student dataset and a logistic regression model for this demonstration.

Step 1: Installation

!pip install evidently==0.4.16 jupyter_contrib_nbextensions==0.7.0 pandas==2.2.1 scikit-learn==1.4.1.post1 jupyterlab==4.1.2 

Step 2: Importing Necessary Libraries

import pandas as pd
from sklearn.model_selection import train_test_split
from evidently.report import Report
from evidently.metric_preset import ClassificationPreset
import json
from sklearn.preprocessing import LabelEncoder
from sklearn.linear_model import LogisticRegression

Step 3: Load and Prepare the Dataset

# Load the dataset
df = pd.read_csv("datasets/student-dataset-v1.csv")

# Encode the target variable
le = LabelEncoder()
df['Result'] = le.fit_transform(df['Result'])

# Separate features and target variable
X = df.drop(columns=['Result'])
y = df['Result']

# Encode categorical variables
X = pd.get_dummies(X)

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Step 4: Train the Logistic Regression Model

# Train Logistic Regression model
lr_model = LogisticRegression()
lr_model.fit(X_train, y_train)

# Add predictions to the training and testing sets
X_train['prediction'] = lr_model.predict_proba(X_train)[:, 1]
X_test['prediction'] = lr_model.predict_proba(X_test)[:, 1]

# Add the actual target values to the datasets
X_train['target'] = y_train
X_test['target'] = y_test

Step 5: Generate and Save the Evidently AI Report

We generate a classification report using Evidently AI and save it in JSON format.

# Generate the classification report
lr_class_report = Report(metrics=[ClassificationPreset()])
lr_class_report.run(reference_data=X_train, current_data=X_test)

# Save the report as a JSON file
lr_class_report.save("json_reports/lr_report_v1.json")

Step 6: Access Data from the JSON Report

Finally, we load the JSON report and extract specific metrics, such as accuracy.

# Load the JSON report
with open('json_reports/lr_report_v1.json', 'r') as file:
    data = json.load(file)

# Accessing accuracy from the JSON report
lr_accuracy_v1 = None
metric_results = data['suite']['metric_results']

for result in metric_results:
    if 'current' in result:
        lr_accuracy_v1 = result['current'].get('accuracy')
        if lr_accuracy_v1 is not None:
            break

if lr_accuracy_v1 is not None:
    print("Logistic Regression Accuracy:", lr_accuracy_v1)
else:
    print("Accuracy not found in the JSON data.")

Output:

Logistic Regression v1 Accuracy: 0.7241379310344828

In this article, we learnt how to use Evidently AI to generate a classification report, save it in JSON format, and access specific metrics from the report. By following these steps, you can efficiently track the performance of your models and ensure they are working as expected in production.

GitHub: https://github.com/Sanjjushri/evidently-ai-poc

Explore more articles on Evidently AI:

1. Monitoring Machine Learning Model Using Evidently AI

2. Integrating Evidently AI with MLflow for ML Model Monitoring