What can moral philosophy teach us about autonomous vehicles?
Making progress on moral dilemmas
As autonomous vehicles (AVs) are deployed at scale, they will encounter traffic scenarios that present moral dilemmas. One kind of dilemma that has received particular attention from philosophers and policy scholars takes the following shape: Suppose an AV is approaching a crosswalk into which a party of five has just stepped. Now suppose further that the AV’s brakes fail. The vehicle can either continue on course, barrelling into the five and killing them all. Or it could swerve and crash into a solitary pedestrian, killing just one. The AV has only two choices and the outcome of each is certain. Harm is both imminent and unavoidable. What should the AV do? What moral values should its decision-making algorithm reflect?
In the debate about this dilemma (call it the ‘AV problem’), at least two popular views have emerged. First, that the AV problem is far too improbable to be of major concern. Instead, mundane situations, like registering a ‘Stop’ sign and braking, deserve more attention. Second, that the AV problem as posed is intractable. Many in this camp contend that it mirrors a popular thought experiment in moral philosophy called the ‘Trolley Problem,’ in which a conductor of a trolley out of control must make a fateful decision: He can continue on course, killing five workers caught unawares on the tracks, or choose to divert the trolley and kill one instead. Writers point out that in spite of the fact that the Trolley and related problems have shed light on various moral theories, they have also generated many conflicting intuitions, and have not been conclusively settled. The same must also be said about the AV problem.
But both these views are deeply unsatisfying. Improbable though they might be, dilemmas like the AV problem are bound to occur at some future point given the many miles AVs will drive. Moreover, there ought to be some principle or rule by which AVs operate in such situations. When human moral and legal rights are at stake, we cannot leave life-or-death decisions to chance.
Driven by these observations, a third view in the debate has emerged (call it the ‘moral AV view’) and is fast gaining popularity. According to this position, when faced with dilemmas like the AV problem, AVs ought to behave morally, or as morality requires. At base, proponents of this view believe that there are indeed right and wrong answers to traditional moral dilemmas like the Trolley Problem, and that those answers can then be applied to the AV problem.
Most typically, the solutions to such dilemmas are in the form of principles or rules which can then be encoded into decision-making algorithms. For example, one proposed principle is ‘Minimize Overall Harm,’ which posits that in moral dilemmas, decision makers should act so as to minimize overall harm. If this principle is correct, then AVs ought to swerve and kill one pedestrian instead of five, and conductors of doomed trolleys must divert them. Another proposed principle is ‘Let Harm Be Done But Do No Harm.’ On this idea, there is a moral difference between letting harm be done and actively doing it — one that explains why it is morally impermissible for a doctor to kill a healthy person and harvest her organs in order to save five terminally ill patients. If encoded in AVs, this latter principle would entail that an AV could not swerve and kill one to save five when the harm is due to an external cause, for instance a second out-of-control vehicle. Proponents of the moral AV view argue that even though it might not be clear for now which moral principle is correct, that does not mean there isn’t a correct one.
While the moral AV view is more compelling than its competitors, it rests on a faulty assumption, namely that AVs are the kind of system into which principles or rules can be programmed. It presumes that AVs are like humans and reason deductively, consciously deliberating from first principles. The idea conceives of AVs as robo-chefs, as following strictly defined recipes to the letter to prepare dishes. But in reality, AVs are not rule-based systems, nor do they operate deductively. There are no recipes in AVs.
On the contrary, AVs are statistical systems, animated by loosely defined logic. Programmers merely outline a high-level architecture of the algorithm and then feed it reams of appropriately annotated data, for instance images of ‘Stop’ signs, or of crosswalks. The algorithm then ‘learns’ to identify ‘Stop’ signs and crosswalks — it teaches itself to pick out the relevant characteristics that constitute each. It might associate, say red-colored octagons, with ‘Stop’ signs. In the most modern algorithms, called ‘neural networks,’ these concepts are represented in billions of finely-tuned mathematical parameters which activate when a ‘red octagon’ is later detected. In this way, formal systems begin encoding semantics, at times even outperforming humans. There remains precious little presence, however, of explicitly defined rules.
So where does all this leave the morality of AVs? A modest conclusion of the above critique is that we ought to build only rule-based AVs so that they can be moral. But the odds of a paradigm shift in design approaches to building AVs are quite low. In the modern era of artificial intelligence, the popularity of statistical systems has eclipsed that of rule-based approaches. Big Data, unprecedented processing power, and technical breakthroughs have enabled neural networks to trouce rule-based systems in a variety of cognitive tasks such as detecting breast cancer and classifying everyday objects. Moreover, the pitfalls of rule-based systems are too deep: We do not understand aspects of vision well enough to capture them in a set of rules. How, for one, does the brain predict the trajectory of an object in motion? If we can’t provide a set of rules for visual processing, we cannot build rule-based AVs which perceive the external world. And an AV that can’t perceive the world cannot drive morally. This does not mean it is impossible to build an algorithm that combines the best of both worlds. The idea of ‘hybrid algorithms,’ which combine the advantages of statistical learning and first-principles deduction, has been championed by the cognitive scientist Gary Marcus. For now, however, it is in its infancy and it remains to be seen whether a ‘hybrid’ AV is even technically feasible.
A stronger conclusion of the critique is that AVs can never be moral agents because they simply cannot obey moral principles. If this is so, then either they should not be deployed at all in present form, or they must be deployed in a fashion that leaves the door open to human moral intervention. Perhaps the system should cede control when it encounters a moral dilemma to human operators, safety drivers, or even appropriately-instructed passengers who have access to the relevant moral principles. The humans could then pull a lever, and the AV would swerve in minimizing overall harm — or they could choose inaction and the AV would maintain its trajectory. The key idea here is that humans must always retain meaningful control in situations where human moral or legal rights are at stake. This logic would apply to other autonomous systems too, like lethal weapons systems.
But a still stronger conclusion also follows: That moral dilemmas like the AV problem should not impede the deployment of AVs. Proponents of this conclusion might argue on consequential grounds; that it might be morally permissible or required to deploy AVs even if AVs behave unpredictably (or immorally) in dilemmas like the AV problem. This is because they would nonetheless save over a million lives per year lost due to preventable traffic accidents. Reports by the National Highway Traffic Safety Administration have shown that over 80% of road casualties are caused by distracted driving, speeding, and driving under the influence. AVs might not be perfect, but they would still prevent those deaths; so the broader consequences would make their deployment moral.
Whichever the conclusion that most resonates with the reader, the core critique — that rules and principles cannot be programmed into AVs — has broader implications for the ethics of self-driving cars, moral philosophy, and legal theory. In the debate so far, much of what is construed as being ‘moral’ is construed from a ‘victim-centric’ approach i.e. what is best for the victims involved in a collision. So when proposing principles like ‘Minimizing Overall Harm’ or ‘Letting Harm Be Done But Doing No Harm,’ thinkers have in mind the human victims of the AV’s actions. But scant attention has been given so far to the agent involved. Relatively little work has investigated whether AVs are even able to act in moral ways. As the ethics of self-driving cars and of artificial intelligence matures as a field, there is much work to be done in ‘agent-centric’ approaches to moral decision making.
