How Much Can We Really Trust You? — A paper on deep neural networks
Towards Simple, Interpretable Trust Quantification Metrics for Deep Neural Networks
The researchers at DarwinAI published a paper about how much we can trust deep neural networks. In order to create any trust at all, we have to first determine what trust actually means in this context and then determine what kinds of measurements are actually “trustworthy”. How much trust do we put in things that are ultimately wrong, but are presented extremely confidently? Alternatively, how much trust do we put in things that are ultimately correct, but are presented hesitantly? For example, if a model identifies a vehicle as a motorcycle with 100% confidence–when the vehicle is actually a truck–are we less likely to trust its next decision? If the model correctly identifies a second vehicle as a truck with 55% confidence, are we more likely to distrust it because it is not confident in its own determinations?
The study presents simple and interpretable metrics to measure trust in deep neural networks based on a thought experiment about trust with respect to confidence to try to better understand where and how trust breaks down. The proposed metrics are by no means perfect, but we hope to push the conversation towards better metrics to help guide practitioners and regulators in producing, deploying, and certifying deep learning solutions that can be trusted to operate in the real world.
DarwinAI, the explainable AI company, enables enterprises to build AI they can trust. DarwinAI’s solutions have been leveraged in a variety of enterprise contexts, including in advanced manufacturing and industrial automation. Within healthcare, DarwinAI’s technology resulted in the development of Covid-Net, an open source system to diagnose Covid-19 via chest x-rays.
To learn more, visit darwinai.com or follow @DarwinAI on Twitter.
If you liked this blog post, click the 👏 below so other people will see this on Medium. For more insights from our team, follow our publication and @DarwinAI. (Plus, subscribe to our letters if you’d like to hear from us!)
