Out to Innovate: Software Biases and Your Projects

The three key takeaways from my talk on Intersectional internet studies, trans-inclusive design, accessibility, design justice, and your projects at NOGLSTP’s Out to Innovate 2019 conference:

1. Software tends to reinforce societal systems of oppression
2. There are ways to do better
3. You can apply these perspectives to your own work

Here’s the slides — followed by a reference list.

Intersectional Internet Studies

Kimberlé Crenshaw on The Urgency of Intersectionality

• Intersectionality refers to overlapping or intersecting social identities and related systems of oppression or domination. Intersectionality: definitions, videos, and links includes videos, and links to papers, articles, interviews, and books, with perspectives on intersectionality from Kimberlé Crenshaw, Patricia Hill Collins, bell hooks, and others.
• Safiya Umoja Noble, Ph.D.’s A Future for Intersectional Black Feminist Technology Studies (in S&F Online) is a good short introduction to intersectional critical thinking about technology.
• The Intersectional Internet (edited by Safiya Umoja Noble and Brendesha M. Tynes) explores a variety of topics. Miriam E. Sweeney’s chapter on “The Intersectional Interface” looks at how gender and race shape the design of anthropomorphized virtual agents (AVAs) and “illuminates the ethical considerations that designers of technology must engage with if they are to engage socially responsible technologies.”

• Malkia Cyril’s Will You Harbor Me? To Fight Police Violence, Demand Digital Sanctuary , from Personal Democracy Forum 2017, discusses how “Simply inserting digital technologies into discriminatory policing without addressing fundamental flaws in its historic design only serves to supersize discrimination, can only reduce community safety and violate the civil rights of the most vulnerable among us.”
• ProPublica’s series on machine bias, by Julia Angwin, Jeff Larson, Surya Mattu, Lauren Kirchner and Terry Parris Jr., was a Pulitzer Prize finalist.
• Dr. Cary Gabriel Costello’s The TSA: a binary body system in practice, on TransAdvocate, is another powerful look at the cis-normativity of TSA’s technologies and systems.

• Safiya Umoja Noble’s outstanding Algorithms of Oppression looks at how search engines reinforce racism. Google’s Algorithm: History of Racism Against Black Women, from Time, is a good introduction. Cordelia Fine’s Coded prejudice: how algorithms fuel injustice reviews Algorithms of Oppression and Virginia Eubanks’ Automating Inequality)
• Alex Dalbey’s Trans People Keep Getting Suspended From Twitter — And They Want Answers, from The Daily Dot, looks at a wave of suspensions that appear to be due to a mass reporting campaign by TERFs. Didi Delgado’s Mark Zuckerberg Hates Black People, Julia Angwin et. al.’s Facebook’s Secret Censorship Rules Protect White Men from Hate Speech But Not Black Children in Pro Publica , and Aaron Sankin’s How activists of color lose battles against Facebook’s moderator army in Reveal, look at other situations where social networks suspend marginalized people who are calling out racism, sexism, or anti-LGBTQ behavior.

Trans-inclusive design

How not to do it

• Mark Wilson’s How Tech Giants Design For Transgender Users–Or Don’t looks at a challenge Sophie Alpert faced after transitioning: “her profile on Airbnb, where she had been a member since 2013, still referred to her with male pronouns and referenced her old name.” Airbnb fixed the problem by updating all references to Alpert’s names and pronouns across all comments.
• Finn Harker and Jonathan Ellis’ Male/Female/Othered: Implementing Gender-Inclusiveness in User Data Collection, from Open Source Bridge, and Sabrina Fonseca’s Designing forms for gender diversity and inclusion are two complementary looks at best practices for asking about gender and pronouns.
• Gopinaath Kannabiran’s Where are all the queers? A research blind spot in Gender & HCI was one of the first papers to look at the heteronormativity of the human-computer interaction (HCI) space. Samantha Breslin and Bimlesh Wadhwa’s Exploring Nuanced Gender Perspectives within the HCI Community, and Olivier L. Haimson and Gillian R. Hayes’ Towards Trans Inclusion in Feminist HCI, both from 2014, helped broaden perspectives.
• Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M. Branham’s Gender Recognition or Gender Reductionism? finds that transgender individuals “have overwhelmingly negative attitudes towards AGR [automated gender-recognition] and fundamentally question whether it can accurately recognize such a subjective aspect of their identity,” and includes recommendations on how to accommodate gender diversity when designing new digital systems. Os Keyes’ The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition shows that “AGR consistently operationalises gender in a trans-exclusive way, and consequently carries disproportionate risk for trans people subject to it,” and includes ideas for alternatives to AGR and how HCI can work towards a more effective and trans-inclusive treatment of gender.
• Gopinaath Kannabiran’s Themself: Critical Analysis of Gender in Facebook; Oliver L. Haimson and Anna Lauren Hoffmann’s Constructing and enforcing “authentic” identity online: Facebook, real names, and non-normative identities; Os Keyes’ Stop Mapping Names to Gender; Alex Ahmed’s Trans Competent Interaction Design: A Qualitative Study on Voice, Identity, and Technology; and Morgan Klaus Scheuerman, Stacy M. Branham, and Foad Hamidi’s Safe Spaces and Safe Places: Unpacking Technology-Mediated Experiences of Safety and Harm with Transgender People look at other important topics from a gender diversity perspective.

Accessibility

Over 90% of the top 1,000,000 home pages have significant automatically-detectable accessibility problems.

• The WebAIM Million looks at the top million home pages from an accessibility perspective. “While this research focuses only on automatically detectable issues, the results paint a rather dismal picture of the current state of web accessibility.” Ethan Marcotte’s The web we broke and Eric Bailey’s Fighting uphill react to this study.
• The W3C’s Web Accessibility Initiative has an Introduction to Web Accessibility, tips on Designing, Writing, and Developing for web accessibility, a summary of the Web Content Accessibility Guidelines as well as the full spec, Authoring Practices, and a lot more info.
• The A11y project: a community-driven effort to make web accessibility easier. Digestible, up-to-date, and forgiving.
• Web accessibility basics by Marco Zehe packs a lot of information into a 50-minute video

Design Justice

• Sasha Costanza-Chock’s Design Justice: Towards an Intersectional Feminist Framework for Design Theory and Practice builds upon the Design Justice Network Principles.
• Costanza-Chock’s Design Justice, A.I., and Escape from the Matrix of Domination, winner of the Journal of Design and Science ‘s “Resisting Reduction” essay contest, begins with a section on #TravelingWhileTrans and ends with “Building a world where many worlds fit.”

• Torn Apart/Separados is “a rapidly deployed critical data & visualization intervention in the USA’s 2018 “Zero Tolerance Policy” for asylum seekers at the US Ports of Entry and the humanitarian crisis that has followed.” Emily Dreyfuss’ Using Library Science to Map the Child Separation Crisis in Wired is a good introduction.

And a few links for the software engineers

My own work focuses on the intersections between software engineering and justice, equity, diversity, and inclusion. Here’s a series of presentations and articles that cover this perspective.f

• Diversity-friendly software, with Shireen Mitchell, SXSW 2017, is a good starting point. As well as the video and slides, the post has copious links on techniques including setting intention, accessibility, flexible optional self-identification, threat modeling for harassment, and algorithmic bias.
• The Techniques for Supporting diversity with a new approach to software wiki pages Tammarrian Rogers and I put together on the Open Source Bridge are a much lengthier but less-annotated (and less-polished) collection of links.
• Transforming Tech with Diversity-friendly software, originally presented at TRANSform Tech, discusses a handful of software projects that apply these techniques, including Dreamwidth Studios, Thurst, Pronoun Island, Heartmob, and a detailed look at the open-source, decentralized, ad-free, anti-fascist social network Mastodon.
• Diversity-friendly software and strategy at TechInclusion Seattle looks at how these techniques apply to O.school, a highly-inclusive sex education software.
• Sex, pleasure, and diversity-friendly software: the article the ACM wouldn’t publish was originally an invited article for the Human to Human issue of XRDS: Crossroads, the Association for Computing Machinery’s student magazine. The student editors loved it. The ACM refused to publish it. This revised version contains excerpts from the unpublished paper, as well as reflections on experience and the opportunities it highlights for software engineering.
• What’s at Issue: Sex, Stigma, and Politics in ACM Publishing, with Alex Ahmed, Judeth Oden Choi, Teresa Almeida, and Kelly Ireland, CHI 2018, looks at the underlying institutional and sociopolitical problems this episode (and others involved in editing the Human to Human issue) highlights. What’s at Issue also proposes starting points for future action for HCI-related research and academic publishing practices.
• Gender HCI, Feminist HCI, Post-Colonial Computing, Anti-Oppressive Design, and Design Justice is an overview and annotated reading list on a range of related topics that aren’t as well known as they should be.

Originally published at A Change Is Coming.