No fair: How can we guarantee fairness in AI?
By Stef Garasto, DataKind UK Ethics Committee
This post is based on discussions during Datakind UK’s data ethics book club on fairness in AI.
What is fairness? And what does fairness mean in the context of data and algorithms? Fair is among the most common 1,000 English words, and we are quick at calling out whether something is fair or not (siblings are particularly good at this). Yet in reality there is no single meaning of fairness. In fact, there are at least 21 different mathematical definitions…and it’s mathematically impossible to meet all of them at the same time. (Arvind Narayanan’s seminar on this topic comes highly recommended; read our summary here).
I disagree
Perhaps, then, it’s not surprising that given a choice of different algorithms, people disagree on which are fair. “Machine Bias”, from Propublica, showcases this challenge. Using the example of software that tries to predict reoffending used in the US, it shows how this is biased against black people. Yet looked at through a different lens the algorithm can be classed as fair. It’s a real life example of “fairness impossibility” problem where the various mathematical formulae cannot be satisfied at the same time.
Human vs machine
Human decision-making isn’t always the hallmark of fairness, either. The bias in the data on which we are training our algorithms comes from us. So should we be comparing the outcome of algorithms to the outcome of human decision-making, rather than to standards of fairness that humans have — as yet — been unable to meet?
Evidence suggests that self-driving cars are better drivers than humans, yet their accidents are put under much higher scrutiny. Why is this? Perhaps because, though we expect people to make mistakes, we also expect them to try to correct them — something that inbuilt biased algorithms can not do? Or perhaps algorithmic biases reinforce existing prejudices while claiming objectivity? Horrifyingly, evidence also suggests the same self-driving cars are more likely to hit people of colour.
Reboot
Maybe this human/algorithm comparison is a red herring. Rather, it may be a question of explicitly and publicly choosing “fairness” as a value to uphold, whatever the method. We can then design systems that abide by this value — whatever our definition. In this world, machines could be called upon to correct for an unjust human world, “fixing” issues that arise due to our human biases.
There is a problem, though. Being in uncharted territory, we would not know how to evaluate whether an algorithm is performing as it should. We have no training data for this. We don’t know whether people who were denied a loan would have repaid it if they were approved. We don’t know whether those refused bail would have committed a crime had they been released. When systems make biased decisions, they produce incomplete information on the outcomes of those affected.
One fairness to rule them all
This is where it may become crucial to define what is fair and what is not.
We could decide to privilege “counterfactual fairness”. Under this system, we would consider a decision to be fair if the same decision would have been made if the person’s “protected characteristics” were different, such as their gender or race. This system is designed to ensure that the protected characteristic has no effect on the final decision. However, it would be near impossible to identify two such cases in real life, since variables like gender and race often have a way of making themselves known via other features, like the place of residence.
As multiple different definitions clash with each other, mathematically speaking, when we pick one we spurn others. For example, predictive parity (e.g. is the proportion of people from group A who are labelled high-risk the same as the proportion from group B?) and equal false positive rates (e.g. the probability of being incorrectly classified as high risk) are incompatible.
Who choses?
With too many definitions of fairness, will people just pick and choose, in some sort of fairness gerrymandering? And who gets to decide — customers, companies, Government? Perhaps, instead of focusing on a single definition, the emphasis should lie on transparency and the ability to seek redress. If we judge a human-made decision to be poor, we know who to blame. It is harder to challenge an algorithm — but it is possible, if creators of AI are open about the definition they are using. For that challenge to have an impact, we need to have a decent legal framework for AI decision-making.
There is much more complexity to “fairness” than any of its definitions, or all of them combined, can convey. Given that algorithms can perpetuate bias at an unprecedented scale, it is crucial that outsiders are able to examine the insides of algorithms that affect us all.
What do you think?
The aim of our new-for-2019 book club is to talk about some of the ethical and social issues in data science. Thanks to the great contributions from attendees. Interested in joining us in London, Edinburgh or online? Get in touch!
