The core issue here isn’t that AI is worse than the existing human-led processes that serve to make predictions and assign rankings. Indeed, there’s much hope that AI can be used to provide more objective assessments than humans, reducing bias and leading to better outcomes. The key concern is that AI systems are being integrated into key social institutions, even though their accuracy, and their social and economic effects, have not been rigorously studied or validated.
Artificial intelligence is hard to see
Kate Crawford

More than just their integration I think what’s interesting is AI’s decisions appear perhaps more rational and warranted to an outsider. For example, as you mention, the ‘beauty contest’ results were based on biased data. However, to someone without intimate knowledge of the problem it’s quite easy to think this was a scientific or rational decision. In other words — it’s a really easy way to overlook discrimination because it *feels* scientifically justified even if it’s just as flawed as a human-implemented method.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.