Why Ethical AI is Hard, and Even Harder Without Causal Inference
Arguing that ethical AI is not an easy subject, is in fact easy. Just think of the two words ethical and AI separately. Both words refer to concepts on which many books have been written. Nevertheless, in this post I will argue that ethical AI is harder without causal inference (CI). And yes, on that subject there have been written many books too.
In a recent article Traag and Waltman (2022) use structural causal models (SCMs), a key component of CI, to define fairness and bias in AI. In one stroke, a handful of competing definitions of bias are tentatively consolidated by applying structural causal modeling. It shows just how much the discussion is still shaping from a technical point of view. SCMs were introduced by Judea Pearl, and are part of the AI, or the technical analysis side of things. The Traag and Waltman article is well written, and well worth a read.
In this post I want to keeps things simple. The goal is to outline ethical AI by outlining the ethical part as well as the AI part, all in the span of a 10 minute read (start your stopwatch!). The ethical part will lean on an presentation given by Rachel Thomas (2023). For the AI part, I will take a practical ethical AI challenge and frame it in the context of SCMs, a Traag and Waltman (2022) light if you will.
I will argue that the ethical part is connected to more general democratic challenges, and that the AI part is connected to challenges in applying technology transparently, and how causal inference can connect the two parts. I will be cutting some corners, and some knowledge of structural causal models will help.
Let’s start with the ethical part.
A short outline of ethical issues in AI
As mentioned above, this section leans heavily on the work of Rachel Thomas (2023). Perhaps her words make the point better; these can be found here. To start, ethical AI is often also coined ethical and fair AI (EFA), I will use this designation from now on.
From the perspective of a machine learning practitioner, EFA often equates explainability. Explainability entails that the score that is generated by an algorithm can be explained, in the sense of how the different features contributed to a specific score. Apart from the fact that a lot of the methods used to explain a score are not without challenges, and sometimes local in nature, explainability is not enough. This will be explained in more detail in the paragraph about actionable recourse.
It will come as no surprise that algorithms, being software, can contain mistakes. Contestability, the ability to identify and have mistakes addressed is very important. Even more so, because the mistakes in algorithms are not confined to software. There are sampling mistakes, data mistakes, modeling mistakes, and there are for sure more ways in which algorithms can make mistakes. Some mistakes might easily be identified once noticed, but contesting an algorithm can be complex and requires knowledge. An example of such complexity will be given in the section Case in point. Making sure that an algorithm does not have the last say, but instead is assisting a decision maker, in case of an obvious error is an important part of contestability.
An actionable recourse is given if it is clear what actions a person can take to alter his or her score. (Let’s assume that the algorithm is uncontested.) In other words, what can I do to change the score an algorithm generated for me? Actionable recourse surpasses explainability because it shows that a score has a clear and actionable relationship with the real world. For example: if you drive a car built after 2010, then your car insurance policy will be lower, because newer cars pose less risk due to technological advancements. An actionable recourse is about creating a common, and acceptable, understanding of the challenge the algorithm is trying to solve. In the ideal case it allows for spill-over of knowledge generated during the construction of the algorithm to its users.
An important question in ethical AI is if an algorithm should be created in the first place. Large scale camera surveillance for example comes at a considerable privacy cost, especially since without this surveillance being complete and total, it will likely just shift the behavior it is trying to prevent to a different location. Another example would be Amazon’s intense monitoring its many production workers, and the alleged algorithm that fires employees (semi-)automatically if performance standards are not met.
Another reason to be very cautious with the application of AI is that at scale algorithms cannot only encode bias, but also amplify or even actively create the outcome that the algorithm was trying to predict by way of feedback loops. Consider for example biased algorithms that in the first place steer police surveillance, and then determine parole in a similar way: people get caught in an algorithmic net that actively creates the outcome that is was supposed to predict, creating new and more biased data. A lively introduction to this topic is given by O’Neil (2017).
What should become clear is that large scale algorithms yield a lot of power. And since the number of people creating algorithms is far smaller than the number of people that are subjected to algorithms, it is also fair to state that AI centralizes power. The question of how does AI shifts power is therefor a valid question. In practice it can easily extend bureaucracy, make avoiding responsibility more easy, becoming another place to point. Paradoxically people impacted have the least power, see the consequences earliest, and often understand what needs to be fixed first. Note how these issues overlap with democratic challenges at large.
Issues of ethical and fair AI overlap largely with democratic challenges at large.
The astute reader will have noticed that I have not directly touched upon the definition of unfairness or bias. The reason for this is that although the notions of bias and unfairness intuitively make sense, it turns out that providing unambiguous definitions is not an easy task. For now I will only provide a taste of the reasoning behind a tentative definition, for the complete definition I will refer to Traag and Waltman (2022).
A case in point
To illustrate a practical application of CI in the context of car insurance, I will use an example I picked up from a newspaper. It so turned out that insurance takers living on the second floor were paying a higher premium than insurance takers living on the first floor. This seems like an odd thing. And although the example might seem like a minor nuisance, it readily translates to more unjustified examples. How might CI help to take this example apart? Because the information in the article was scarce, as this kind of information usually is, I will use some speculation to see what might have transpired.
On the face of it, the model seems to suggest that living on the second floor is associated with a higher accident risk. But what is exactly the implication of this statement? Tentatively formulated as an SCM, it is stating that living on the second floor causes a higher risk, this statement would render the following SCM.
Would moving a person to the first floor really influence his or her driving behavior? Not likely. So the relationship is not causal. More likely is that the relationship is created by one or more confounding factors, which might be the actual causes. The following SCM likely offers sounder reasoning.
In this SCM it turns out that social economic status (SES) is a confounding factor, and the ownership of a new car is a mediator between SES and accident risk. SES influences both home and car ownership, for this reason it is called a confounder. People with a higher SES are probably more likely to buy first floor suburban homes, and also newer cars. Newer cars, aided by technology, pose less risk. The association between living on the second floor and accident risk can be explained by SES. There might be more confounders, but lets keep this simple.
From the above analysis one could conclude that someone living on the second floor who also owns a new car should likely not pay a higher premium because the association between living on the second floor and accident risk disappears conditional on new car. This kind of reasoning used to make me nervous, but not anymore. Technically speaking, we closed a backdoor. This procedure is a standard tool in structural causal modeling. And there are more tools to analyze causal models: it turns out to be all probability mechanics. The good news is that this kind of reasoning does not necessarily involve mathematics, it can be done visually and reliably with a piece of paper and pencil. A nice introduction is given by Pearl and Mackenzie (2019).
The gist is that the feature living on the second floor is not a valid cause of accident risk. The feature is probably going to be neutralized in the model if there is a new car feature. Importantly the new car feature offers a valid actionable recourse. A linear model will likely throw living on the second floor out in the presence of new car, but a tree model is more greedy more ambiguous. In a feature importance plot the feature will likely pop up, the tree will likely maintain the feature to no real effect. This example should make clear that just throwing in features and making these explainable is not enough. Hoping that the wrong features get somehow automatically neutralized in the case that the therefor necessary features are present, is not a good strategy. It is doubtful if explainability will solve this. Modeling without thinking can lead to bias pretty easily.
At this point you might think: all good and well, but all of this case is mostly based on your own speculation. You would be right in thinking that way. The great thing about causal inference is that with the dataset used to create the original model, I have a good chance of actually testing my speculative conjectures. I am able to corroborate or refute these conjectures using data and causal inference. And that is progress.
Conclusion
The goal of this post was to outline ethical AI by outlining both the ethical and the AI part in 10 minutes, by cutting some corners. And to show that although both ethics and AI are large fields, adding causal inference can actually clear things up.
I started with outlining the ethical part, and showed how ethical issues are largely connected to democratic issues at large. This poses big concerns. As for the AI part, first of all good definitions are required. Recent developments show that causal inference might just be cut out for the job. Note that this post focused on EFA in the context of models for tabular data and that there are more EFA issues outside of this context.
Apart from confounders, there can be also colliders in structural causal models. Colliders can be connected to sampling issues, rationalizing another concern in EFA. And there is more. The insurance example was illustrated from the perspective of how an individual policy taker can obtain an actionable recourse. Causal inference, through structural causal models, also provides the means to reason about the impact of algorithms or policies on a larger groups of people by way of counterfactuals, rationalizing yet another concern.
As an addendum, but an important one, McGilchrist argues that the human brain with its two halves is very much prone to abstract and schematic thinking; the AI side. It is very hard to think on an abstract level, and at the same keep in focus the things that one should value, or consider sacred: the ethical side. He offers this mechanism as an explanation of the meta-crisis: the crisis of crises. His views help to explain how the notion of EFA is not only challenging for data science, but perhaps for humans at large. A lecture of McGilchrist in the Oxford Darwin College Lecture Series is recorded you-know-where.
It feels like we are just getting started.
References
McGilchrist, I. (2019). The master and his emissary: the divided brain and the making of the Western world. New expanded edition. New Haven: Yale University Press.
O’Neil, C. (2017). Weapons of math destruction. Penguin Books.
Pearl, J., & Mackenzie, D. (2019). The book of why. Penguin Books.
Thomas, R. (2023). AI and Power: The Ethical Challenges of Automation, Centralization, and Scale. Moving AI ethics beyond explainability and fairness to empowerment and justice. Downloaded on 14/07/2023 from https://rachel.fast.ai/posts/2023-05-16-ai-centralizes-power/
Traag, V.A., & Waltman, L. (2022). Causal foundations of bias, disparity and fairness. ArXiv, abs/2207.13665.
