What is Transparency in AI?

What does it mean for a machine learning algorithm to be “transparent”?

Catherine Yeo
May 20, 2020 · 3 min read
Photo by eberhard grossgasteiger on Unsplash

When looking at fair and ethical algorithms, transparency is a key concept often brought up in discussion. But what exactly does it mean for a machine learning algorithm to be “transparent”?

Like “fairness” and “privacy”, it sounds important and useful, but the concept of transparency is quite ambiguous and is worth exploring in more detail.

Definition

  • What is the algorithm doing?
  • Why is it outputting this particular decision?
  • How did it know to do this?

Types of Transparency

  • The developer who builds the system
  • The deployer who owns and releases the system to the public
  • The user of the system

For example, developers are hired to build a personalized recommendation system for online shopping. Amazon (or any large e-commerce company) deploys it, which is used by a regular member of the public.

Then, with these parties in consideration, there are many different types and goals of transparency (as delineated in this paper), including and not limited to:

  • For a user to understand what the system is doing and why, which helps the user get a sense of predicting what it might do with different actions
  • For society to overcome fear of the unknown and build trust in technology
  • For a user to understand why a particular decision was outputted, which provides a check that the system worked as expected (this is critical in applications like criminal sentencing)
  • For experts to test and monitor the system
  • For a deployer to retain users, because transparency would allow the user to feel more comfortable and so they keep using the system

In the last example, while the user receives the explanation, the deployer is the true beneficiary of the system’s transparency. When we evaluate transparency, we must consider who the system is for versus who is benefitting from the system.

We can also differentiate between global and local transparency. Global interpretability explains the overall system, while local interpretability explains a particular decision or classification. When we measure transparency, we must decide which mode of transparency provides a “better” explanation in the specific context.

Conclusion

Related Work

Adrian Weller. “Transparency: Motivations and Challenges”. First submitted to Workshop on Human Interpretability in Machine Learning at ICML 2017. https://arxiv.org/pdf/1708.01870.pdf

Thank you for reading! Subscribe to read more about research, resources, and issues related to fair and ethical AI.

Catherine Yeo is a CS undergraduate at Harvard interested in AI/ML/NLP, fairness and interpretability, and everything related. Feel free to suggest ideas or say hi to her on Twitter.

Fair Bytes

Sharing byte-sized stories about fairness & ethics of AI

Catherine Yeo

Written by

CS @Harvard | I write about fairness & ethics in AI/ML for @fairbytes | Storyteller, hacker, innovator | Visit me at www.catherinehyeo.com

Fair Bytes

A Medium publication sharing byte-sized stories about research, resources, and issues related to fairness & ethics of AI

Catherine Yeo

Written by

CS @Harvard | I write about fairness & ethics in AI/ML for @fairbytes | Storyteller, hacker, innovator | Visit me at www.catherinehyeo.com

Fair Bytes

A Medium publication sharing byte-sized stories about research, resources, and issues related to fairness & ethics of AI

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store