Probing & Prediction Beyond the Label
Why Artificial Intelligence needs Anthropologists
What’s the last predictive model you learned about or tried to use? Have you ever thought about what you were actually predicting?
In digital transformation, businesses are seeking to balance the automation of their labor force with human-in-the-loop augmentation. When a business takes risks, they’re putting their good reputation at stake, a reputation built on high quality standards, customer relationships, and their own corporate social accountability.
As Data Scientists and Software Developers, our work is very precise and requires great attention to detail. When stakeholders ask important questions about the relevance and impacts of our work, these questions rarely, if ever, reach our ears. How many of us have thought about these kinds of broad questions, or would even be able to answer them?
KPMG is a global network of professional firms providing audit, tax and advisory services. They are using IBM Watson OpenScale to incorporate trust, transparency and explainability into AI to the benefit of their stakeholders. Tools like IBM Watson OpenScale are only as effective as the questions we are asking of our Data Scientists and Developers. IBM Developer Advocates created a tutorial that scores a German Credit Risk model. Credit models predict how likely someone is to repay money lent to them. Based on this likelihood of repayment, a person is denied a loan or approved with interest rate based on their score. Why did the developers who created the tutorial below think people in the “male” category are more likely to get loans?
How does someone’s gender affect their ability to repay a loan? Could insight into this question make a financial product more widely available?
If length of time someone has lived at their current address affects someone’s ability to repay their loan, then why does being a Foreign Worker have little impact on this prediction? Could this insight result in a better metric that makes for more accurate predictions?
While the skills that Data Scientists and developers bring to an organization are highly valued, a real challenge that businesses face in adopting AI technologies is collaboration with the people who actually know the business (and the customers). Tools like IBM Watson OpenScale are an improvement but as the example above shows, they still require a lot of developer expertise.
A significant consequence of alienating subject matter experts from everyday development practices is in how it ultimately limits how models are built and trained. These models drive predictions that not only affect a business’ reputation with its own stakeholders, but can have significant social impacts as well.
Do you think the people who trained the German Credit Model above are likely to be denied a loan? What “sex” do you think they identify as?
The challenge of designing human-centered AI is at the heart of Data Ethics and AI for Good efforts, although these groups have a high concentration of expertise in high-precision fields like Computer Science and AI. Social scientific disciplines, such as anthropology and sociology, are often not part of a traditional “hard” science curriculum. Methods like Ethnography require long-term training and apprenticeship, developing a highly-skilled craft which applies extensive knowledge of peoples and cultures into actionable insights that organizations may take up in meaningful ways.
Anthropology, or the study of what makes us human, has been described as “the most humanistic of the sciences and the most scientific of the humanities.” Today, anthropologists are very active in a number of interdisciplinary organizations addressing the broader social impacts of AI’s digital disruption, a few of which are described below.
Ethnographic Praxis in Industry Community (EPIC)
Originally founded to better engage academic anthropologists and anthropologists working in industry (a.k.a. “applied” anthropologists), today their conferences attract professionals beyond User Experience and Human-Computer Interaction. In addition to an annual conference, EPIC offers online workshops for qualitative research methods and excellent articles discussing the impacts of technology from an anthropological perspective.
“We’re building new ways of thinking, knowing, and working at every stage of the product cycle. EPIC is the rare conference where passionate, creative people from diverse contexts are creating these new ways of thinking and doing. Ethnographers bring essential concepts and tools to this brave new cognitive world.” — Mark Burrell, Design Director, IBM Watson Health
Data & Society
The non-profit research institute Data & Society offers a regular email newsletter, blog, and an excellent podcast that makes social science research accessible to an industry audience. President and founder danah boyd earned her interdisciplinary PhD (UC Berkeley School of Information) working with Mimi Ito, an anthropologist who studies online communities.
“The issues that Data & Society seeks to address are complex. The same innovative technologies and sociotechnical practices that are reconfiguring society — enabling novel modes of interaction, new opportunities for knowledge, and disruptive business paradigms — can be abused to invade people’s privacy, provide new tools of discrimination, and harm individuals and communities.” (from the Data & Society About page)
AI NOW Institute
An interdisciplinary Anthropology-aware research institute based out of NYU, AI Now draws on a broad fields of expertise to bring attention to AI’s social impacts. They host live-streamed symposia on topics such as The Growing Pushback Against Harmful AI with “leading lawyers, organizers, scholars, and tech workers, all of whom have engaged creative strategies to combat exploitative AI systems across a wide range of contexts, from automated allocation of social services, to policing and border control, to worker surveillance and exploitation, and well beyond.”
“[D]ata sets are always functions of human history. They reflect who we are. So in many ways, we have to think about these problems as social problems first, not technical. And so for us, that means bringing a lot more disciplines into the room. That means sociology. It means law. It means anthropology, philosophy and history.” — Kate Crawford, AINow Founder and Microsoft Principle Researcher (via NPR)
EASA Applied Anthropology Network
The European Association of Social Anthropologists (EASA) Applied Anthropology Nework (AAN) bridges practical research in anthropology and ethnography with audiences beyond the academic walls. Their annual conference, “Why the World Needs Anthropologists” (WWNA) explores applications of anthropology beyond basic research.
“Why The World Needs Anthropologists is a provocatively titled annual showdown, bursting out of the intersections of human-centered, critically oriented academia and innovative creative industries.” (WWNA Website)
Anthropology + Technology Conference
Launched by anthropologist Dawn Walter, the Anth+Tech conference seeks to make an interdisciplinary space for dialogue around socially-responsible AI including how to build it, creating cross-disciplinary teams, and how business and society can both benefit.
AI can provide unparalleled opportunities, but without the correct processes, it can also have a detrimental impact on people’s futures and a business’s reputation. The need for ethical standards and the perspective of anthropologists has never been more critical. (Anth+Tech Conference website)
Thinking about a Master’s in Data Science or Computer Science?
For a more flexible skill-set, consider an applied anthropology program. The University of North Texas offers a part-time, online Master’s degree in Applied Anthropology. Students work on real projects with actual clients to learn qualitative, quantitative, and ethnographic research methods through applying academic anthropological theory to real-world problems. Instead of a narrow focus on the flavor of the year, students can learn how to incorporate statistics and computer science into mixed-methods research to provide actionable insights and relevant solutions.
Contribute to Center for Open Source Data and AI Technologies at IBM
Lastly, check out our open source projects dedicated to infusing AI with trust, transparency, and explainability. While these projects are built and maintained by IBM’s Center for Open Source Data and AI Technology (CODAIT), we welcome contributions from everyone. Diverse perspectives ensure the open source AI tools we build together create value for more people. We invite you to co-create with us!
