A generational reflection on Virginia Eubanks’s “Automating Inequality”
My parents grew up poor. My Dad told me stories about his childhood bedroom, which was a broom closet with a small mattress in his parents’ laundry business backroom. Some of his first memories were waking up on his back, staring at the flickering pull-string light bulb, with the smells of cleaners and mold and stale water. As a small child, his parents would send him out for the day to capture pigeons in downtown Portland so they could have fresh eggs for breakfast. Similarly, my Mom told me stories of playing in the fields on her parents’ farm with her dozen siblings. Everyone shared one bathroom, and on hot summer days, she was trucked miles from home to pick berries on her neighbors’ farms for 8 hours with her nine sisters. She later learned that this helped her Dad pay the utilities. My notion of poverty was therefore one of generational interdependence: my parents relied on their parents for food, shelter, and safety, and their parents relied on them for labor.
These stories of their childhoods were hard to reconcile with how I knew them as a child. They had both found their way successfully through school. My Dad, being Chinese in the 1960's, was initially routed by Benson High School counselors in Portland toward the trades, but he had a teacher one summer who helped him with his English and pointed him toward college. My Mom, a child of evangelical Christians, started at a bible college in Los Angeles, but became disillusioned with a religious education. My parents met each other at Oregon State University in the 1960’s, my Dad spending some time in the Vietnam war to earn G.I. Bill benefits to finish college, while my mom worked and saved at the local grocery store. They graduated, got married, and moved to Ontario, Oregon, where my Mom taught primary school, my Dad ensured beans were safe to eat. I was born shortly after.
Because my parents both had stable professions, I didn’t grow up poor. I had a room, which I shared with my younger brother, and it wasn’t full of cleaners or mold. I didn’t have to work each summer from the age of 6, or venture out alone to hunt game for us to eat. I had toys, food, and play time with my brother and neighborhood friends. And as my parents’ wealth grew, our homes got larger, my toys got more expensive, and my quality of life rose. We weren’t wealthy—combined, my parents might have made $60,000 annually, and that didn’t stretch far after they got divorced—but it was still far above the $13,000 poverty line of 1990. This money, the schools it gave us access to, and the computer it allowed us to buy, are the only reason I found my way to college to study Computer Science. The National Science Foundation’s investment in Research Experiences for Undergraduates was the prime reason I found my way to graduate school. And the Earned Income Credit was the only reason I could pay my mortgage and feed my family of three as a poor doctoral student making $25,000 a year in Pittsburgh. All of these social welfare programs—or as I like to think of them, social investments—added up to me earning a Ph.D., a tenured professorship, and the upper middle class life I now live.
Of course, throughout my childhood, I had no concept of the systems that made my family’s rapid rise in quality of life possible. I don’t think my parents did either. We were just navigating the systems as they were designed, unaware that they were shaped by particular values. For my parents, these were values such as, women can be nurses, teachers, or secretaries, but nothing else; Chinese American men should do trade work; serving in the military is how people of color should earn access to college. And for me, in the wealthy suburb of Portland that my parents clung to, the values were things like all of the graduates of West Linn High School should go to college; AP exams are a path to success; it preferable to love math, science, language arts, and fine arts, rather than the trades. These are the values that shaped the systems that shaped our lives. And while some of these values limited my parents’ opportunities, they helped our family keep a step ahead of the poverty of my grandparents.
Virginia Eubank’s Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor, examines poverty, and the systems and values that underlying efforts to manage poverty. But unlike my personal examination above, which is a story of escape from poverty, Eubank’s considers the stories of those trapped in poverty, and the modern digital systems that continue to entangle them. The book, therefore, is the story of what my life might have been now had my parents not had the timely benefits of public assistance and outstanding teacher in the 1960's.
The book begins with a history of poverty and data in the United States. Eubanks details the birth of “scientific charity” in the late 19th century, when my great grandparents first emigrated to the United States from Denmark and China. She describes a political movement to convert values-driven safety nets to more efficient “evidence-based” safety nets:
“… social scientists fanned out across the United States to gather information about poor people’s sex lives, intelligence, habits, and behavior. They filled out lengthy questionannaires, took photographs, inked fingerprint, measured heads, counted, children, plotted family trees, and filled logbooks with descriptions like “imbecile,” “feeble-minded,” “harlot,” and “dependent.” (p. 22)
These notions of data took on a semblance of objectivity, but hiding underneath these structured records were the usual American racism, classism, and ableism. My grandparents managed to avoid most these systems because of extreme frugality, child labor, pride, and hidden subsidy (e.g., farm subsidies); though my grandmother, orphaned by the death of her mother, was swept up into the foster care by an orphanage, where she was almost certainly documented to justify the aid given her orphanage.
Eubanks then chronicles a century of political whiplash of public generosity and fiscal conservatism. The Great Depression brought increased support through the New Deal, helping my grandparents to stay afloat and begin saving, as they worked as farmers, post office workers, World War I cooks, launderers, and mothers. At the same time, the New Deal and LBJ’s war on poverty introduced the idea of the “deserving” poor, which hardened class and racial biases into policy and law. Social movements fought for welfare as a right and won; but white sympathy for Black poverty declined, and Reagan’s vision small government arrived to quash anti-poverty efforts.
Eubanks argues that what ultimately replaced this whiplash was a faith in the neutrality of technology to efficiently and “fairly” distribute shrinking resources:
“By the 1980’s, computers collected, analyzed, stored, and shared an extraordinary amount of data on families receiving public assistance… fraud detection programs were carefully programmed and launched. Databases were linked together to track recipient behavior and spending across different social programs. The conflict between the expanding legal rights of welfare recipients and weakened support for public assistance was resolve by a wave of high-tech tools.” (p. 34)
The premise of these systems as they were, and as they are today, was that a combination of statistics, algorithms, and data collection could objectively determine who was deserving and who was not. Eubanks, however, views them as little more than a modern and amplified version of the same punitive and paternalistic systems that have tried to manage the limited resources of poverty remediation rather than actually addressing the causes of poverty.
After this history, Eubanks turns her attention to three cases. The first concerned Indiana’s early 2000’s efforts to reduce public spending on food stamps, Medicaid, and cash benefits. Indiana’s stated goal was to reduce bureaucratic overhead through computing:
“The Daniels administration instated that moving away from face-to-face casework and toward electronic communication would make offices more organized and more efficient. Even better, they argued, moving paper shuffling and data collection to a private contractor would free remaining state caseworkers to work more closely with clients.” (p. 47)
What actually happened was that IBM created an organizational nightmare, erecting new bureaucracies, creating dehumanizing disarray in case management, and a system that treated people in need not as people in states of crisis, but as data records, divorced from context, to be judged as deserving or not. The result was an increase in complexity in rules, a compounding of unintended side effects, and ultimately, a dramatic increase in the number of people denied aid. The state achieved its goals in some ways, spending less money on its safety net—it just did it by helping fewer people and traumatizing them in the process.
Eubanks then turns to recent Los Angeles efforts to manage the limited supply of public housing. Here, the case concerned its new “Coordinated Entry System”, which sought to match the newly homeless to short term support. The system, which once again gathered immense amounts of data in order to make “optimal” matches, appeared to do little more than create new barriers to accessing limited resources:
“For Gary Boatwright and tens of thousands of others who have not been matched with any services, coordinated entry seems to collect increasingly sensitive, intrusive data to track their movements and behavior, but doesn’t offer anything in return. When I asked T.C. Alexander about his experience with coordinated entry, he scoffed, “Coordinated entry system? The system that’s supposed to be helping the homeless? It’s halting the homeless. You put all the homeless people in the system, but they have nowhere for them to go. Entry into the system but with not action.” (p. 114)
This was an example of prediction used not to address the underlying root causes of homelessness by expanding a needed resource, but to try to better allocate a limited and shrinking resource based on questionable judgements of need.
The third case Eubanks presents is the Allegheny county’s child protection service’s effort to optimize the case worker time toward complaints most likely to be “legitimate” concerns. Here, the idea was to gather data about families, their children, and their communities in order to determine whether a complaint about the family was likely to lead to further investigation upon inspection by a screener, and ultimately an intervention, such as taking a child from their guardians. Eubanks reports how faith in the models ended up warping manager’s and screeners’ perceptions of ground truth:
“We all tend to defer to machines, which can seem more neutral, more objective. But it is troubling that managers believe that if the intake screener and the computer’s assessments conflict, the human should learn from the model. The AFST, like all risk models, offers only probabilities, not perfect prediction. Thought it might be able to identify patterns and trends, it is routinely wrong about individual cases.” (p. 142)
A later evaluation of the system showed that the system’s ability to accurately predict which referrals would be further investigated was slightly increased, especially for children of color, but these trends attenuated slowly after the intervention, mirroring the troubling feedback loop in the quote above showing how these models shaped human judgement rather than the other way around.
Eubanks ultimately argues that applications of predictive models and algorithms to the management of poverty interventions are an abandonment of empathy building. After all, building models from data means stripping away the context of people’s lives, under the presumption that data alone, even carefully gathered data, can accurately represent who “deserves” aid. Her argument is that by following this path, we not only further dismiss people’s humanity, but expand the degree to which they are surveilled, reduce their autonomy by eliminating human judgement and empathy, and by taking attention away from the root causes of poverty.
And take attention away it did. I lived in Allegheny county, just above the poverty line, and had no idea that AFST was being built while I lived there. As a young parent in a lower-class, high-poverty borough, CPS was probably investigating the Black families in my town, but not our family. I had middle class extended family that lived in near Los Angeles, and visited them frequently. Why didn’t I know about the public housing crisis down the freeway from my family’s homes? And my ex wife’s family in Indiana, in their seven bedroom estate, never mentioned the dismantling of anti-poverty programs happening in their state when they talked politics; they only talked about rising taxes and freeloaders.
While I was busy being unaware of growing disregard for poverty, debate about how to manage declining anti-poverty resources continued. Eubanks clearly has no faith in information technology to remedy poverty. However, the rise of data science in my own academic communities felt the opposite, not only questioning her blanket dismissal of data science for optimizing limited government aid, but often advancing the tools and techniques for doing it. My peers and colleagues’ arguments were abstract claims of improved fairness, transparency, and reliability, and an implicit dismissal of the value of human judgement and morality. My community’s attention, then, was less on poverty itself and more on the part of poverty that seemed computable: how to allocate resources.
Of course, this is precisely Eubanks’ critique: her book argues that systems of fixed or declining resources, no matter how efficiently and transparently distributed, cannot be fair or just, no matter how advanced our technologies get. She argues that bias in poverty doesn’t emerge from methods of the allocation of resources—it emerges from biased decisions about who does and does not get our abundance of resources. After all, these cases occurred in the United States, the wealthiest country in the world—there is enough food, shelter, and health care for everyone. A select few have just used their power to decide that some people don’t deserve it.
At some abstract level, I don’t personally see the cases for and against technology as in conflict. I believe our country needs a much stronger safety net, like those in wealthy countries in Europe, and that these would greatly relieve the pressure on deciding who is and is not deserving. That is not a technological problem, but a political, social, and moral one. But I also believe that when resources are limited, it may be possible, with immense care given to the individual, organizational, and moral considerations of these systems’ design, implementation, and deployment, that we might build software that achieves transparency. It won’t be fair or just, but if the code is open, at least it will be clear that it’s not.
In practice, however, I agree with Eubanks, that giving our scarce technical imagination to this lesser vision of justice is highly problematic. I’d much rather see future generations of data scientists and software engineers who commit their energies to public service to spend their time identifying hidden inequities and injustices rather than inadvertently perpetuating them by reinforcing austerity. Data scientists should be our scouts, on the front lines of poverty, helping us notice the failures in our ever more complex systems, giving voice to those caught in our tangled, frayed, and fragile nets.
This history of poverty, and these recent efforts to try to manage our declining investments in eradicating it, have been invisible to me, and to my family. I didn’t realize it at the time, but my family was in a race to outpace the grasp of poverty and modern poverty management systems, as my country’s wealthy whites stripped away support, and tightened their grip on the families left behind. That technology has been used to amplify this inhumane vision of America is just a symptom of a much deeper commitment to making my family’s unlikely rise out of poverty in the 20th century an impossibility.
