How to Stop Computers Being Biased
The bid to prevent algorithms producing racist, sexist or class-conscious decisions
By Madhumita Murgia
One of the first major court cases over how algorithms affect people’s lives came in 2012, after a computer decided to slash Medicaid payments to around 4,000 disabled people in the US state of Idaho based on a database that was riddled with gaps and errors.
More than six years later, Idaho has yet to fix its now decommissioned computer program. But the falling cost of using computers to make what used to be human decisions has seen companies and public bodies roll out similar systems on a mass scale.
In Idaho, it emerged that officials had decided to forge ahead even though tests showed that the corrupt data would produce corrupt results.
“My hunch is that this kind of thing is happening a lot across the United States and across the world as people move to these computerised systems,” wrote Richard Eppink, the legal director of the American Civil Liberties Union (ACLU) in Idaho, which brought the court case.
“Nobody understands them, they think that somebody else does — but in the end we trust them. Even the people in charge of these programs have this trust that these things are working.”
