Experiments on XGBoost: Part 2
Accelerating SHAP Value Calculation with GPU
Ever wondered how XGBoost behaves when run on CPU and GPU? Well, this story tries to answer this question by examining different outputs generated by running various combinations of an XGBoost classifier.
GPU support on XGBoost is not something new. Thanks to this feature, we can accelerate our model training process and generate results in a fast way. With this fresh story series Experiments on XGBoost on Hypatai, we will focus on the effect of running XGBoost models both on CPU and single GPU. We already examined the effects in terms of model training and performance evaluation metrics in Part 1. In this story, we will continue with studying GPU accelerated SHAP value calculation.
Experiments we did were run on a huge dataset consisting of ~1.7 m rows, ~400 columns in the training set, 20 % of it being used for evaluation. Shap value calculations which we will soon talk about were made on the test set which has ~350K rows. Tasks were completed by constructing XGBClassifiers using version 1.3.1, keeping each hyperparameter (and the seed) constant except the ones which are compared in upcoming tables.
Calculating SHAP values makes us feel like a superhero when it comes to explaining complex machine learning models, yet it is a tough, but certainly a must-have task in real-life data science projects. It can also be a computationally costly task, but thanks to XGBoost’s GPU support for SHAP acceleration, it is not much of a problem now.
Using GPU acceleration on SHAP value calculation is as easy as adding the below two lines to the subsequent regular calculation script :
The above table gives metrics related to what we will explain in this story as well as the ones which were explained in Part 1. Therefore let’s just focus on the ones that are relevant to Part 2, specifically, results in the color blue corresponding to the prediction. As shown above, columns show the results of making predictions on CPU or GPU. Notice that there is also a parameter called the predictor, which is set to cpu_predictor on CPU and set to gpu_predictor on GPU (We focused on rows in the matrix that are related to the model training, and the metrics in red in the previous story).
In order to compare the SHAP value calculation time on CPU and GPU, we must select the same model. To do that we compared results on model 1 where SHAP value calculation lasted for 38.84 secs on CPU and 16.66 secs on GPU. A similar comparison is executed for model 2 is as well, as shown in the table.
So what conclusion can we make after executing this experiment?
- No matter the model trained on CPU or GPU, time gain in computing SHAP values on our test set are similar for the two models. (Reduction in SHAP calculation time is similar both for model 1 and model 2).
- Setting gpu_predictor to cpu_predictor (or vice versa) for the same model does not affect the predicted value (If we saw such a case it could be a bug, so this behaves as expected). Note: predicted values corresponding to the first record in the test set are shown in the first table.
- We would save a lot in terms of response time if we had an API to explain the reasons of our model output to our customers by using XGBoost’s GPU support on SHAP value calculation.
The story series Experiments on XGBoost: Part 1 and Part 2 ends here. See you at new stories on Hypatai. ✋
