Event Recap: Automating Inequality Book Discussion

TechEquity Executive Director Catherine Bracy and author Virginia Eubanks discuss her book, Automating Inequality.

Last Thursday, along with Code for America, we co-hosted a discussion of the book Automating Inequality with the author, Virginia Eubanks. Automating Inequality was the third book selected for the TechEquity book club and it’s a critical read for anyone interested in technology for social good.

About 100 people gathered to hear Eubanks discuss how digitizing social programs can have disastrous consequences, unintended and intended, for the people who need social services the most. You can watch recorded livestream here.

Recorded livestream of the discussion.

We kicked off with Christa Hartsock, engineering lead at Code for America and cofounder of Logic Magazine, explaining how Code for America works on improving government processes and the challenges they face in doing so.

Catherine Bracy, Executive Director at TechEquity, and Virginia Eubanks discussed the three case studies of the book and the implications they have for building digital infrastructure for social programs moving forward:

1. Computerized system for processing welfare eligibility in the state of Indiana. The state of Indiana wanted to cut costs and make the system more efficient, but the resulting computer system lost applicants’ documents, severed ties between caseworkers and the people they served, and denied one million applications in the three years it was used. There was such a negative impact on people in need of services in Indiana that they eventually terminated the program.
2. Coordinated Entry System (CES) for housing the unsheltered in Los Angeles. To optimize the available services for housing the 58,000 unsheltered people in LA, the city commissioned the CES to rank applicants based on perceived need. To rank applicants, caseworkers are collecting deeply personal information on a massive scale that can easily be utilized by police departments to surveil and criminalize people for living on the streets. Though the system has worked well to match the available limited housing and supportive resources, it has captured data on thousands of people who end up in the system without access to any of the services promised.
3. Predictive model for a child welfare agency in Pittsburgh. In Allegheny County, PA, the Department of Human Services uses a predictive algorithm to determine which children are more or less likely to become victims of abuse. In doing so, they take human bias and codify it into their system. One of the factors that goes into the model is called re-referral, where people report families for abuse and neglect. People report Black and biracial families for abuse and neglect three and a half times more often than they report white families. This racial bias gets embedded and obscured within the system, resulting in a disproportionate targeting of Black and biracial families.

“At their worst, these tools can act as an empathy override, where they allow us a relief valve,” says Eubanks. She tells us that in the face of numbing digitization, we need to stay present and uncomfortable with inequity and its surrounding problems.

Eubanks argues that system engineering has become the seductive option to solve for social problems, but optimizing systems does nothing positive without increasing tangible resources at the same time. For example, Georgia State University has been applauded widely for successfully using predictive analytics to prevent students from dropping out of school. What has been left out of the story is that at the same time as this analytics initiative was implemented, they also increased their staff and available appointments by 500%.

Here are some key questions that Eubanks has for anyone looking to build a social impact tool:

1. “Is the tool you’re building going to increase the self-determination of poor working people?”
2. “If we built the tool for anyone but poor people, would they tolerate it?”
3. “When you’re looking at using more analytics, is it coming with more resources?”
4. “Are you working closely with the people most directly impacted by these systems?”

As we work to address the Bay Area’s deep inequalities in housing and labor rights, we need to keep Eubanks’ critiques and suggestions in mind in order to make real solutions that work for those who need them most. If you want to join in on the fight, sign up for our newsletter to get involved. Big thanks to Virginia Eubanks for joining us and to Code for America for co-hosting this event!

We’re uniting tech workers to create a more equitable economy. Join us!

We believe the tech industry, built on the internet — the most democratizing communications platform in human history — can and should contribute to broad-based economic growth that benefits everyone.