We live in an age of black-box and white-box models. On the one hand, black-box models have observable input-output relationships but lack clarity about their inner workings. This is typical of deep-learning and boosted/random forest models, which model extremely complex situations with high nonlinearity and input interactions.
White-box models, on the other hand, like linear regressions and decision trees, have behavior, features, and relationships between influencing variables and output predictions, but are often not as performant as black-box models, i.e., lower accuracy but higher explainability.
The ability to explain to stakeholders why certain predictions are made is required by the majority of machine learning systems. When selecting a suitable machine learning model, we frequently consider accuracy vs. interpretability.
- Accurate: ‘black-box’
Black-box models, such as neural networks, gradient-boosting models, and complex ensembles, are often highly accurate. In addition, these models don’t provide an estimate of the importance of each feature on the model’s predictions, nor do they make it easy to understand how the different features interrelate with each other. - Weaker: ‘white-box’
Complexity is not always modeled well by simple models such as linear regression and decision trees (i.e. feature interactions). However, they are much easier to understand and interpret.
In the real world, however, both types of models have their time and place. Not all decisions are the same, and developing interpretable models is difficult, if not impossible, in some cases, such as when modeling a complex scenario or a high-dimensional space, as in image classification. Even in less complex scenarios, black-box models typically outperform white-box counterparts due to black-box models’ ability to capture high nonlinearity and interactions between features: a multi-layer neural network applied to a churn detection use case. Despite their superior performance, black-box models have several drawbacks. The first disadvantage is a lack of explainability, both internally and externally, to customers and regulators seeking explanations for why a decision was made.
The second disadvantage of black-box models is that there may be a slew of unseen issues affecting the output — such as overfit, spurious correlations, or “garbage in / garbage out” — that are impossible to detect due to a lack of understanding of the black-box model’s operations. Another drawback of not spending enough time understanding the reality beyond the black-box model is that it creates a “comprehension debt” that must be repaid over time through difficulty in maintaining performance, unexpected effects such as people gaming the system, or potential unfairness.
Some Other useful articles:
