Interpretable and ethical self-driving cars
co-authored by Hlynur Davíð Hlynsson and Lisa Schmitz
We’re living in exciting times where the potential of technology seems almost limitless. It’s easy to get caught up in the hype and forget that our actions have consequences.
An ethical dichotomy
In a simplified view of ethics, there are two main kinds of decision-making policies: utilitarian and deontological.
From the utilitarian point of view, the ultimate goal is to maximize happiness or minimize suffering. The goal, through action or inaction, is to create outcomes that have this measure of “goodness” maximized. According to this, the end justifies the means.
On the other hand, a deontologist strictly forbids certain actions. Some things are good or bad in themselves, no matter the consequence. According to this, inflicting harm is never justified, even if it saves lives.
A classic example is the trolley problem:
There is a runaway trolley barreling down the railway tracks. Ahead, on the tracks, there are five people tied up and unable to move. The trolley is headed straight for them. You are standing some distance off in the train yard, next to a lever. If you pull this lever, the trolley will switch to a different set of tracks. However, you notice that there is one person tied up on the side track. You have two options:
1. Do nothing, and the trolley kills the five people on the main track.
2. Pull the lever, diverting the trolley onto the side track where it will kill one person.
Which is the most ethical choice?
The utilitarian answer to this problem is to pull the lever and save five people at the expense of one. If the choice is between two outcomes, then choosing the one that minimizes pain and suffering should be the obvious one: the lever should be pulled, and a net amount of four lives will be saved. This line of reasoning sounds good but can come with a few issues.
Another trolley problem
This problem has been a part of the admission test for Icelandic medical students. It turns out that the reasoning above is frowned upon, as acting to save five people is considered the same as murdering the one person.
Consider a hospital that has, let’s say, five patients who will die unless they get an organ transplant. One day a traveler with perfectly matching organs and no friends or family comes in for a routine checkup. A utilitarian surgeon might be tempted to pull the metaphorical lever and harvest the traveler for his organs, saving his other five patients.
This transplant scenario was first described by Judith Jarvis Thomson and puts the situation into perspective. The organ transplant example is of higher relevance for real-world scenarios than the trolley problem. In a short time frame, more people might live if we choose to kill the traveler. But it is predictable that we will lose our trust in hospitals if we expect the possible fate of being harvested for body parts. This will cause fewer people to seek help at hospitals which will, in the long run, lead to a greater overall loss of lives.
The key concept here is trust. We trust that there are some strict rules that ultimately benefit everyone and protect you as an individual. The rules hold even though there are single cases (like the organ transplant case), where following that rule set does not immediately result in the best outcome — in terms of the total amount of saved lives. In order for you to trust the rules, they have to be understandable and not made on a case-by-case basis by a “black box”. Designing the rule set like this guarantees that people can trust a hospital to act in their best interests. And with this trust, we are able to establish allows hospitals to save millions of lives every year.
Self-driving cars: moral machines?
Unfortunately, it seems that we have not yet completely developed this attitude towards self-driving cars. Instead of focusing on a deontological system that rejects harmful action, many discussions focus on single scenarios and therefore encourage a utilitarian system that tries to maximize goodness in single case scenarios at the individual’s expense.
MIT has an online platform called The Moral Machine where they present visitors with scenarios and ask them how to resolve them. In each of these scenarios, people must decide who lives and who dies. It’s provocative, to say the least. After judging, the visitor is presented with some metrics about their choices. For example, if they prefer to protect the passenger of the car or if they prefer protecting people with a higher social value.
Try it for yourself: http://moralmachine.mit.edu/
As examples like the trolley problem show: the majority of people tries to save as many lives as possible in single case scenarios. However, if we follow those seemingly intuitive choices we most certainly will end up with a system that is untrustworthy. Individuals must be able to trust the systems that govern their lives. They must feel like they are protected if they obey the rules (e.g. traffic laws), otherwise the system is doomed to fail.
Exploiting ethical self-driving cars
Another flaw of a utilitarian system is that it is easy to exploit. Let’s assume that we have a decision-making system in a car that values children’s lives higher than that one of adults. According to the outcome of the Moral Machine evaluations, this corresponds to peoples’ basis of decision-making.
Consider a world with utilitarian self-driving cars. We have a person, the murderer, who wishes to kill another person, the victim. The murderer knows that his victim takes the same path to work every morning at around 09:00 with his self-driving car. It just so happens, that there also is a school located at one point of the victim’s way to work. The murderer now positions himself in front of the school, waits for his victim and his self-driving car. When he sees the victim approaching, the murderer pushes a child — who was just on her way to school — onto the street in front of the victim’s car. The only way to save the child is to make a sharp turn to the left: directly into a concrete wall. There might be a 99% chance the driver will die in this maneuver. However, it’s a child’s life vs. an adult’s life. The system chooses to sacrifice the driver.
Reliable ground rules
While a deadly exploitation might not just occur as easily as this, it remains a possible scenario. And we cannot create a system that enables people to take a person’s life by endangering a child.
Therefore we need a system of rules, that protects human life equally and places a high value on following the law. This way, we can create a trustworthy and predictable entity. In the case of self-driving cars, it is certainly not as easy as just a handful of rules that form the decision process but we suggest the following points as the core values of such a system:
- Avoid situations where a human life is lost
- Obey traffic laws
- Drive along predictable paths
These rules should determine a solution for each of the cases in the Moral Machine. The only moral decision that the computer should be allowed to make is to avoid killing people if possible. But the complex, non-interpretable decision-making should stop there.
Conclusion
By all means of the importance of discussing ethics with AI, approaching this topic by using something like the moral machine is controversial. The focus on single, situational cases creates a distraction from the overall goal of preventing all the accidents that are momentarily caused by human drivers. Moreover, those cases are often highly constructed and their actual probability of occurrence is low.
With all the dramatizing of AI in media where this technology is mostly portrayed as a negative — if not even destructive — force on humankind, it is important that we design AI systems not as a moral black box, but as predictable, trustworthy software.
