Encoded laws, policies, and virtues: the offspring of artificial intelligence and public-policy collaboration
Note: the draft article is below. The final article was published in the Cornell Policy Review here.
Policy-makers must engage with technologists in the development of artificial intelligence machine learning systems. If they do not, then they’re implicitly adopting a utilitarian approach to the results these systems produce, focusing only on the fairness of the outcomes and not the means that achieved those ends. There is a major problem with this hands-off approach. Technologists often, consciously or unconsciously, encode laws, policies, and virtues in machine learning systems that make decisions. If policy-makers are not engaged in the creation of these systems, they are ceding the balancing of these variables to technologists who are not experts in the trade-offs inherent in public policy development. In fact, technologists focusing on system optimization may have no idea they are encoding laws, policies, and virtues at all.
The media has frequently reported on bias in artificial intelligence tools (e.g. racial bias, gender bias, Facebook, Google, and HP, among others). These issues are especially apparent in machine learning systems that can automatically and unintentionally encode biases. Machine learning systems, and especially deep-learning systems (a sub-set of machine learning), require large data-sets for training. These data-sets are frequently drawn from real-world data and, as we know, the real world has a significant number of biases. If these biases are not corrected before training, they pass through to the output of the machine learning system.
A Google machine learning system delivered ads for job openings to women that paid less than job openings targeted towards men. Why might this be? Currently in the United States, women are paid ~.75-.80 cents for every dollar men earn. One theory is that the Google system was trained with real world data where this gender bias was baked in, so the results reflected real-world bias. It wasn’t the machine’s fault — it was just sending men and women jobs it thought were the best matches based on the training data it was given. And you could blame the technologists who created it, but their skill is in creating effective software, not correcting gender bias, which is the specialty of a public-policy professional. (Compounding the problem, most AI researchers are male — the sea of dudes/white guy problem — and AI job ads tend to target male applicants, perpetuating the problem). Nevertheless, this type of issue is potentially lurking in many machine learning systems and we need new cross-disciplinary collaboration to solve it.
The public policy profession needs a new professional specialization in machine learning and data science ethics. If policy professionals are not engaged with technologists in the creation of machine learning systems, society will be forced to take a utilitarian approach towards these systems — focusing on the fairness of outcomes. This is a costly and inefficient approach, because unfair outcomes will need to be challenged in court or through administrative procedures. Machine learning complicates post hoc remedies because these systems don’t have a decision-tree like traditional algorithms. They are more statistical than algorithmic; more ‘black-box’ than auditable. (And systems that do have auditable decision trees simply aren’t as effective). While researchers are hard at work on this issue, machine learning systems can’t yet explain to you why they made a decision. So it’s entirely possible these systems could be making the right decision most of the time, but for the wrong reasons. And the incorrect decisions could be inexplicable without a thorough audit of the data used to train them.
If an unjust machine learning tool is challenged in court, the costs of a lawsuit may just be the beginning. If a machine learning system is found to be unfair, its data will need to be audited and cleaned, and the system will need to be retrained, adding considerable expense. A preferable approach is for policy and machine learning experts to work together on the creation of these systems to ensure they encode the laws, policies, and virtues we want reflected in their outputs. Fairness is a problem that should be fixed on the front-end, not policed on the back-end through legal liability.
AI researchers are examining the tradeoffs involved in creating fair machine learning systems. Scientists at the Department of Computer and Information Science at the University of Pennsylvania conducted research on how to encode classical legal and philosophical definitions of fairness into machine learning (see 1,2,3). They found that it is extremely difficult to create systems that fully satisfy multiple definitions of fairness at once. There’s a trade-off between measures of fairness, and the relative importance of each measure must be balanced.
Replace the word ‘fairness’ with ‘justice,’ ‘liberty’ or ‘non-partisanship,’ and we start to see the challenge. Technologists may be unconsciously codifying existing biases by using data-sets that demonstrate those biases, or could be creating new biases through simple ignorance of their potential existence. Technologists should consciously remove these biases and encode laws, policies, and virtues, (shortened for our purposes to ‘values’), into machine learning systems. These values can be mathematized, but there will always be tradeoffs among different values that will have real-world impacts on people and society.
The same Univeristy of Pennsylvania scientists found there can be a cost to fairness in machine learning. The penalty to a machine learning training rate from encoding fairness can be mild to significant depending on the complexity of the model. The cost of encoding virtues (and the benefits) must be balanced against the potentially lower costs of systems that are optimized without regard to values or bias. For a machine learning system that delivers ads for clothing, encoding values may not be worth the cost. But for a system that determines eligibility for food stamps or advises on criminal sentences, the cost of bias is severe and likely worth the expense of encoding values. Technologists can make these tradeoffs, but this is not their area of expertise. Balancing the tradeoffs between different values is the realm of public policy professionals. They must engage with technologists if we want machine learning systems to make decisions that reflect society’s laws, policies, and virtues.
A real world example illustrates why it is so important for public-policy professionals to engage before machine learning systems are created, rather than relying on post hoc remedies. Imagine that a U.S. state wants to eliminate gerrymandering by using a machine learning system that can draw legislative districts automatically. There are a number of different policy goals that could be encoded for drawing fair districts, like: seeking representation proportional to the number of votes for each party state-wide; creating districts that keep population centers and communities intact; generating districts with equal population; ensuring racial and partisan fairness; maintaining compact, contiguous districts; et al. The manner in which these values are encoded in the machine learning system will make significant differences in how the congressional districts are drawn and the resulting balance of political power in the state.
As you can imagine, everyone from politicians to lawmakers to lobbyists would have a strong interest in working with technologists to ensure their equities are represented in the construction of such a system. They wouldn’t just leave it up to the technologists to build the system, and then rely on legal challenges if they didn’t like the outcome. Interested parties would get involved as the system was being built. The stakes are simply too high for the politicians, lawyers, and policy experts to leave these value trade-offs to technologists that don’t specialize in this field. In this gerrymandering example, policy experts’ equities are obvious because the impacts on businesses, organizations, and people are so large.
So why aren’t public-policy experts clamoring to engage with technologists developing machine learning systems that have public impact? And why aren’t machine learning companies demanding engagement with the public-policy community?
Let’s extrapolate the gerrymandering example to the full range of machine learning applications. These systems will be used for making choices that impact nearly every decision in our lives — ranging from credit eligibility for businesses and individuals; to the ability to secure government services and public assistance; to determining whether you are a security risk; to eligibility for insurance. The machines will start out as advisors to humans, but as they become more trustworthy, people will outsource some complex decision making completely to machines. The EU was concerned about this possibility and it drafted rules that would ban significant autonomous decision making about EU citizens unless appropriate protections, laws, or consent are in place. This type of precautionary principle is laudable, but we should work to make these types of laws unnecessary. An industry code of conduct is one solution, but such an agreement is moot without people who can actualize it. What we really need are experts who can build the right kinds of systems, rather than laws that could stifle innovation and prevent the development of machine learning decision making tools.
Governments need to be aware of these concerns and public-policy professionals should be assigned to work closely with technologists who are developing machine learning systems where bias could have significant impacts on public services. Conversely, technology companies need to be aware of the implicit value judgements they are building into machine learning systems. The smart companies will hire their own public-policy experts to help data-scientists and technologists uncover hidden biases and ensure the right balance between relevant laws, policies and virtues. Woe be to the company or government that neglects this critical element of machine learning system design for a tool with significant public impact. They are likely to find themselves in a court of law, explaining their negligent discrimination. And if U.S. corporate machine learning systems can not demonstrate fairness in their creation, they may be unable to access the E.U. market due to its cautious autonomous decision making rules.
Academic institutions also need to adapt. University machine learning curricula need more policy-based instruction so technologists are aware of these concerns. And we need a new subspecialty in the public-policy profession: the data and machine learning ethics analyst. These experts will have a solid grounding in both public-policy and technical fields. They will be able to educate machine learning developers and data scientists about the value judgements inherent in their systems, assist in removing bias from data-sets, and advise how to encode values to remove bias.
If public-policy and artificial-intelligence can spawn this offspring, it will give us a chance to encode values in our machine learning decision making tools. AI’s have the potential to take inevitable human biases out of decisions and significantly reduce the bias in critical decisions, but this requires public-policy professionals and technologists to work together on ensuring values are built into machine learning systems during development. The values we encode in our artificial intelligence systems should be chosen affirmatively and consciously so they reflect the laws, policies, and virtues of our entire society.
Values are the most important choices we make as human beings. Our intelligent machines should reflect the importance of those values.
—
Want additional reading about this topic?
The AI Now summary report discusses many issues around ethics and fairness and has an excellent section on bias and social inequality.
The Fairness, Accountability and Transparency in Machine Learning Workshop is a hub for information. They have an extensive bibliography as well as a set of Principles for Accountable Algorithms and a Social Impact Statement for Algorithms
The Berkman Klein Center has an excellent video by Jonathan Zittrain on Openness and Oversight of Artificial Intelligence, and additional videos on ethical issues in AI.
Aaron Roth blogs about issues of fairness in machine learning at his website Adventures in Computation.
Mike Loukides has a good article on AI ethics covering similar issues.
There is a Fairness, Accountability and Transparency in Machine Learning community.
—
Opinions expressed are my own and do not represent the views of the U.S government or any other organization.
