It’s Time the US Created an AI Protection Agency to Regulate AI… Here’s How
A proposal for leveraging the institutional strengths of Congress, courts, and federal agencies while fostering greater civic engagement with impacted communities
Algorithmic accountability is a hot topic. As of 2021, there were 131 proposed bills mentioning AI in Congress, up from just two in 2012. Why? Well, AI/ML applications increasingly pose public risks. Public risks are broad threats to human health or safety outside our individual understanding and control, such as nuclear technology, environmental pollution and mass-produced objects.
Whether it’s social media platforms’ use of reinforcement learning-based recommender systems, or public agencies’ and companies’ use of risk prediction systems in healthcare, education, finance, or criminal justice, the public risks of AI/ML systems ought to be managed and allocated in a comprehensive, yet fair and democratic way. Rather than relying on the dubious ability of tech corporations to promote the common good, why not “ensemble” our various government institutions to make this goal a reality?
This post outlines a holistic proposal for regulation of AI/ML in the context of the American political and legal system. Its strength lies in its balanced approach that goes beyond abstract ethical principles to leverage the relative strengths of legislatures, agencies, and courts in order ensure that technological innovation is aligned with human interests and democratic values.
One warning, however: I’m not a lawyer, just a concerned citizen.
Filling in the Gaps Posed by High-level AI Ethics Principles
Translating high-level ethical principles into actionable AI/ML policies and developing public accountability mechanisms remain urgent tasks in AI ethics. Fortunately, legal systems possess an array of institutions designed to bridge these gaps and fairly distribute the public risks generated by AI/ML.
Courts in the US and UK possess what legal scholar Peter Cane refers to as a “rich tapestry of responsibility concepts and practices” established through case law, also known as common law or judge-made law. Courts can thus extend the useful doctrines, concepts and practices evolved from hundreds of years of experience with e.g., defective products, negligent doctors, and polluting corporations to AI/ML systems.
A Three-pronged Regulatory Proposal for AI/ML
This section adapts the proposal in Matthew U. Scherer’s 2015 paper, which I recommend to anyone interested in the legal complexities involved in AI/ML regulation. The power of Scherer’s idea is its balanced approach harnessing the different institutional strengths of legislatures, agencies (i.e., ministries), and courts.
Scherer starts by recommending legislatures pass something akin to an Artificial Intelligence Development Act (“AIDA”) that lays out the general normative principles and values guiding AI/ML systems.
Importantly, the AIDA would establish an agency dually responsible for setting federal AI/ML policy and certifying system safety. I’ll refer to this agency as the AI Protection Agency or “APA,” for short. The APA would be staffed by data scientists, machine learning researchers, social scientists and other experts familiar with the challenges posed by complex sociotechnical systems. More on this later.
Why Regulatory Holism Is Good: Institutional Diversity
Government bodies possess different institutional competencies. AI/ML regulation should ideally leverage the relative strengths of legislatures, administrative agencies, and courts to achieve a synergistic effect.
Although often criticized for lacking technical expertise, democratically-elected legislatures have a certain normative legitimacy and are therefore well-situated to propose general guiding principles and values for AI/ML development. The sensitivity of the legislature to public opinion is especially important given the social justice critiques of AI/ML as perpetuating structural inequalities across racial and social boundaries. Besides having the power of the purse (budgets), legislatures can also delegate authority to specialized agencies.
Agencies employ domain experts working in the public interest to translate high level normative ethical principles into actionable and concrete policies guided by technical knowledge. This domain knowledge is important when AI/ML is applied in criminal justice, healthcare, education, or financial contexts. Agencies can thus be tailor-made to regulate specific industries and social problems, such as commerce, energy, or environmental protection.
Agencies provide ex ante forms of AI/ML regulation by publishing industry standards for their development, including setting acceptable ranges for fairness metrics, conducting independent safety and reliability investigations of particular AI/ML systems, and also disseminating information about the risks of discrimination and bias. Ideally, the AI/ML experts employed by the agency are relatively immune to capture by big tech.
Courts rely on judges to interpret the laws (the statutes enacted by legislatures) and juries to ascertain legal liability and fair compensation for defective or negligent AI/ML design (a key part of what is known as tort law). Courts also possess specialized fact-finding, discovery and evidentiary procedures. The results of such procedures could, for example, be used to formulate AI/ML accident and safety reports similar to those in the aviation industry. These reports can help spur research into new AI/ML failure modes and develop corresponding AI safety mechanisms and procedures.
The drawback of courts, as opposed to legislatures and agencies, however, is their “reactive” or ex post nature that requires a harm be done to either an individual or group before courts can intervene.
Proportionality & Liability: Certified vs. Non-certified Systems
The AIDA would apply different legal liability regimes to certified and uncertified AI/ML systems, as determined by the APA. Liability means legal responsibility for compensating victims of harm or injury, which is the essence of tort (i.e., injury) law, as opposed to criminal law. This restorative or balancing function of the courts is known as corrective justice.
According to Scherer, developers of APA-certified systems should only face limited liability. Those harmed by an AI/ML system would thus need to prove negligence in its design or operation, which can often be difficult. Non-APA certified systems, however, would be held to a higher standard of strict liability — e.g., a standard traditionally applied to manufacturers of defective products — which drastically lowers the burden of proof and incentivizes AI/ML developers to ensure their systems are safe, fair and reliable. This can deter bad AI/ML developers from entering the market.
This approach reflects the oft-used regulatory principle of proportionality, whereby AI/ML systems posing greater public risk (e.g., high-stakes “black-box” systems) are subject to greater regulatory scrutiny and legal accountability. For instance, liability rules can be tailored to achieve a more equitable allocation of the public risks of AI systems by forcing those who use or design them negligently to compensate victims.
Ideas for APA Certification & Auditing Procedures
Continuing with the hypothetical example of the APA, here I outline some data science procedures that may help set federal pre-certification standards or audit standards to maintain limited liability certification.
Simulation and generative modeling will undoubtedly play a key role in certifying safe AI/ML systems. Synthesizing datasets with known biases (selection bias, differential measurement, data missingness, etc.) can contribute to robust and fair system design. One “fairness by design” approach would examine an AI/ML system with respect to several formal fairness metrics at the group and individual levels. Generative adversarial networks (GANs) could simulate datasets with meeting certain statistical standards of “fairness,” which can then be used to train certified AI/ML systems or serve to benchmark algorithm performance. Domain expert collaborations may also estimate causal path specific effects of common legal and data science decisions on predicted risk scores. For instance, AI/ML certification could require predicted risk scores to be causally insensitive to certain “unfair” choice paths.
Additionally, ideas from transfer learning could be used to assess group-level fairness metrics in the presence of distribution shift, as often occurs when AI/ML systems trained in one geographic or demographic region are later deployed in others.
Another audit technique for high-stakes predictions might involve measuring and reporting the degree of predictive multiplicity for an AI/ML system. Beyond some range or threshold, the APA could revoke the system’s certification, thus exposing its developers to greater liability and financial burden for imposing greater public risk.
In medical and criminal justice contexts, unexpected results from fairness investigations could spur further sociological research and public policy proposals to examine root causes of social inequality, medical treatment and policing disparities. Lastly, the APA could set standards for when deployed algorithms should be checked for degrading performance to reflect changes in medical treatments, policing, public policy, and administrative data collection. In short, sociotechnical systems require data scientists to adopt a more holistic approach to auditing and testing.
Responsibility via the Professionalization of Data Science?
If the public risks embodied by AI/ML in high-stakes medical and public health domains are any indication, data scientists may need to professionalize in order to better discharge their legal and moral duties related to public-facing AI/ML design and development. This might require taking on formal fiduciary duties to act in the best interests of the public, similar to the way lawyers must work on behalf of their clients, doctors for their patients, or parents for their children.
A recent paper about the use of AI/ML in medical contexts has suggested a role of algorithmic stewardship by a mixed group of doctors, patients, ethicists, data scientists and safety and regulatory organizations to provide ongoing monitoring and oversight. But data scientists may need additional practical and academic training in order to succeed in a public-facing stewardship role, e.g., human subjects research, criminal justice system and administrative data collection practices.
Further, the role of courts in setting standards for legal liability needs to be further investigated. Standards for negligence and medical malpractice are typically based on professional standards or customs, yet no such professional standard of care exists for data scientists. Black-box methods and undocumented data collection and analysis procedures also pose clear problems for assessing the impact or foreseeability of AI/ML pipeline and design decisions on predictions. Courts’ convergence on appropriate standards of care will take considerable time.
General AI/ML Regulation Beyond the FTC and FDA?
These considerations are merely a starting point for a much larger public discussion. Many important legal and technical details remain to be clarified. Currently, no such general AI/ML certification agency exists in the US. While the Federal Trade Commission is the de facto agency investigating cases of algorithmic discrimination and bias, it’s limited by its consumer focus. And the Food and Drug Administration (FDA) deals with medical AI/ML applications. One might argue these agencies are spread too thin.
Nevertheless, there is precedent for this proposal. The proposed 2021 EU AI Act is based on the concept of proportionality and classifies AI systems in healthcare, criminal justice, and in financial contexts as posing “unacceptable risk,” “high risk,” or “low or minimal risk.” High-risk systems must meet higher standards of transparency, traceability, accuracy, human oversight, and robustness than those deemed low-risk. Some applications of algorithmic risk assessment, such as social credit scoring, may present so much public risk as to justify being banned altogether. Lastly, regarding the role of the courts, the European Parliament is considering proposals for reforming legal liability rules in order to better manage the public risks of new and emerging AI technologies.
In the US, the proposed Algorithmic Accountability Act of 2019 makes a similar appeal to proportionality of regulatory scrutiny based on risk levels.
Ideas for Advancing Public Interest Data Science
I am worried by the fact that fewer and fewer AI PhDs go into public service. According to Stanford’s 2022 AI Index report, roughly 60% go into industry, 24% into academia, and only 2% into government. This means industry needs and the profit prerogatives of financial markets increasingly dictate the the allocation of human labor to AI/ML applications and research. We instead need to foster a feeling of civic engagement, particularly in young data scientists, whose predictive models may be applied to their fellow citizens.
If there is to be an agency dedicated to certifying AI/ML systems as safe, reliable, and adhering to the normative guidelines and ethical principles embodied in the AI Development Act, the agency staff should be composed of the best and brightest young data scientists. Here are a couple ideas to start.
Suggestion 1: Cancel Student Debt for AI Protection Agency Staff. The AI Development Act should include a provision that cancels student loan debt so that new MA and PhD data scientists can take jobs within the AI Protection Agency immediately and not have to worry about crippling student loan payments. This will help balance the asymmetry between data scientists in industry and government due to much higher salaries.
Suggestion 2: Create a non-profit “Data Science for America.” This is based on the example of the non-profit Teach for America, which sends “high-performing” young university graduates to teach in historically under-resourced areas. The AI Development Act should include a provision to provide service opportunities for young AI and data science graduates to work with local city and state governments, particularly in underserved areas. At the end of 1–2 year service stints, these volunteers would have their student loan debt forgiven.
