Chapter 4: Regulation
When it is Needed and How It is Done
By Mikey Fischer and Shreyas Parab
What is regulation? When do we need it? How is it done?
Imagine that we have a semi-autonomous agent, something like an Alexa but much smarter. A new technology, such as quantum computing, is used to create a virtual assistant that learns through reinforcement learning. This new virtual assistant has superhuman capabilities to do tasks that go far beyond current tasks that Alexa can do. Instead of just playing music and setting alarms, this Alexa can book the perfect vacation, set up the perfect date, or find the best price for something online. With more computing power and AI, instead of being reactive, it could be proactive. Instead of booking an appointment with a doctor, it could directly diagnose.
This virtual assistant has the power to do great good. But it could also cause harm unintentionally by it taking actions without any human telling it to do so. President FANG (Facebook, Apple, Netflix, Google) or is a recently elected president. President FANG asks you to determine if this new super intelligent virtual assistant should be regulated? And if so how? President Fang has experience in the technology sector before going into public service and strongly believed in the power of technology and “Silicon Valley”-esque innovation to positively impact the world.
He acknowledges that technology has its pros and cons. If a disaster or accident was caused by a mistake as a result of the virtual assistant, for example a missed appointment, a wrong medical diagnosis, or a misunderstood command that caused death, who would be held responsible for the virtual assistant’s mistake? Is the company who created the autonomous virtual assistant liable? Or is it the person that used the virtual assistant? Should Congress have a virtual assistant to testify about the disaster? If the Facebook hearings taught the public anything, it was perhaps the lack of understanding regulators, specifically people in Congress, have on even basic technical issues.
Pros and Cons of Regulating Virtual Assistants
The pros of having a powerful virtual assistant is that society will operate more efficiently. Any form of automation, such as automated tellers, allow for people, and societies being more services to more people at less of a cost. With automation, the quality of products goes up, there is higher product output, employees (that remain employed) are more productive, and people are free from the mundane.
With a very powerful virtual assistant, the worry is about unintended consequences of such a technology. With a virtual assistant, how do we deal with the capriciousness of human desire. Reinforcement learning requires some period of adjustment as the algorithms are slowly self-adjusting to better meet the needs and wants of humans. Unfortunately, the needs and wants of humans are constantly changing to the point that perhaps by the time the algorithm starts to determine what that need is our desire has already changed.
Regulating can allow for more certainly in deploying an uncertain system.
The cons of regulating virtual assistants include that developers and technologists will now have more personal or corporate liability that might prevent innovation from taking place. While responsibility and liability are extremely important for the protection of others, many critique that overburdening amounts of ill-planned regulation prevents necessary technologies and solutions from being brought to market. Even more so, by limiting the abilities of virtual assistants, we might augment datasets in order to not include certain key pieces of information that help inform reliable and accurate representations from the virtual assistant.
For example, in regulating virtual assistants, we might struggle with how to deal with the capriciousness of human desire. Reinforcement learning requires some period of adjustment as the algorithms are slowly self-adjusting to better meet the needs and wants of humans. Unfortunately, the needs and wants of humans are constantly changing to the point that perhaps by the time the algorithm starts to work that the needs of the controller have already changed. Regulating these would allow for more certainly in deploying a system. In the context of reinforcement learning, such systems need to interact with an environment to be able to collect significant amounts of data before they can be useful. Giving the developers a clear idea of what is allowed when deploying a system would give them certainty about how they can legally, and safely, deploy the system. This would reduce the uncertainty, lower the risks of developing the software, reduce the cost of the software to the end user, and bring about the agents to the market more quickly.
This technology, since it is using reinforcement learning, often develops a set of “behaviors” that could prove problematic over time. Even more so, it affects the transparency of any situation where suddenly the decision-makers are no longer humans who can respond to inquiry, but computers’ whose algorithms often operate in a blackbox of weighing systems and nodes that are far too complex to disentangle.
From Regulating Virtual Assistants to Regulating More Broadly
We go from the example of the virtual assistant to the more general case of why society has developed regulation in certain forms. There are many reasons for why society chooses to regulate.
Market Failures
Free markets are powerful, yet there are problems that they are not able to solve. One of the most prominent is the formation of monopolies. When one company becomes too powerful it is able to control the market by producing goods or services that lack a viable substitute which leads to a higher price or lower quality product. The Sherman Act made monopolies illegal and the Clayton Act, which made illegal certain business practices. Examples include price discrimination based based on who is buying the product. Deals with companies that put conditions on sales, for example tying products together. Microsoft tied their internet browser, Internet Explorer, a product that people didn’t want, to their operating system, Windows, that people did want. Limits mergers and acquisition restrict competition. A person from being a director of a two or more competing corporations. that are known to lead to the formation of monopolies.
The goal is to overconcentration of economic power over a market. This rationale is often used most prominently in the case of preventing market manipulation and regulation of financial markets. Economic power that concentrates to a small group of holders puts not just public markets at risk, but due to the intertwining of public and private markets in modern America would adversely topple the stability of institutions. Economic power quickly translates to political power and in order to preserve the stability of economic and political institutions, regulation exists to protect those institutions and the individual American citizen. Oftentimes considered one of the most controversial rationale for regulation and a driver in the political differences between the leading two American political parties on the role of government, preventing the concentration of economic power lies at the intersection of selflessness and selfishness for regulators. They want to give everybody a fair chance of getting a piece of the pie, but they also regulate in order to make sure that there is actually even pie available to take.
Collective action and desires
Why can’t consumers make the correct choices that take into account the wellbeing of others? The reason is that market actors can produce negative externalities and so the benefits gained by the individual making a decision might have a negative impact on a third party. For example, companies can have a negative carbon impact that will affect the environment or the creation of secondhand smoke while manufacturing products can have repercussions in public health.
Systemic and Systematic Risks
There are wider risks posed when multiple independent actors fail at the same time. Some industries are particular interconnected so that when one fails, it might signal that other are soon to fail too. We take for example the banking industry. If one large bank fails, it can lead to a chain reaction that causes the other banks to fail, which can in-turn lead to wider economic destruction. One way to prevent these types of risk that can have important economic consequences for a government is to regulate interconnected and important industries.
Professional Conduct
Beyond the domestic reasons to regulate, there are broader international reasons to regulate. In the United States, we often don’t consider the power and place of international organizations as much because oftentimes this is exactly how the United States exerts influence on other countries across the world, especially emerging economies that promise new opportunity to be realized. Yet in fact, these international organizations build international norms as to what the terms of engagement are. For example, when the international community decides that X is right and Y is wrong, they use the collective power of multiple nations to enforce that belief… usually a strong enough force that a dissenter could not rise up against or dispute. For example, international treaties help get all stakeholders in the world at the same table on issues that affect every country such as climate change or human rights. Although these are incredibly difficult geopolitical environments that are even more slow-moving than domestic regulation, they help set the norms of what is expected from countries. Countries who adhere to these regulations now accept the terms and the relationship building that results, but oftentimes it is hard to enforce the violation of these terms because all a nation needs to do if it wishes to not participate is withdrawn.
There are also concerns about the conflict of intertemporal utilities. Intertemporal utilities is essentially the process in which humans change their behavior and wants in different points of time. For example, if you were to capture the wants of a person at any given point of time and then frame them up next to the the next million instances, you would find a clear discrepancy in what people want. To simply state what you might already know, humans don’t know what they want and when they want it. This takes place in society in cases like tobacco addiction or consumer finance. Humans would be quick to smoke cigarettes or vape without considering the future impact of these actions because of human desire, but the government might rationalize regulating because they’d want to prevent the massive public health outcomes that come with millions of smokers. This kind of cost to society would be massive, not only in lost productivity, but also in massive healthcare costs and demographic shifts that would threaten the strength of a country.
Another rationale for regulation that is often used both for rhetorical purposes and to better govern for ALL. In the United States, the minimum voting age is 18, meaning that tens of millions of American exist that are not represented in government. Legislators might regulate in order to protect classes or groups that might not have traditional representation in government, but are considered a “protected class” such as children or others who are not considered capable of making decisions. This kind of rationale is what allows the Federal Trade Commission to regulate advertising to children because they believe that children are not at the point where they can reasonably consent to have their information collected online and require adult intervention to help decide for them.
We also regulate more effectively when it comes to incredibly scarce resources. For example, scarce resources like land and nature necessitate agencies like U.S National Parks Service in which the government takes ownership of land and manages it just because they want to ensure that it is protected because once it is used up, it will be lost forever. Arguably, this is one of the most powerful rationales for government because so much about American history in all its triumphs and especially in its flaws do we think about scarce resources and how the government distributes it. For example, how the United States stripped Native Americans of their land and then decades later gave some back. There is so much power in land and so it is very important to be critical of the rationale for regulation around land. Oftentimes the most underrated government agency is the US Bureau of Land Management, but they control almost ⅛ of the US’s landmass (247.3 million acres!) a staggering amount of responsibility.
The final reason why regulation might take place is perhaps the most frustrating rationale for innovators which is prudential rational. This rationale focuses on the inability to measure or define the “goodness” or “badness” of any decision based purely on the consequences of that action. Also known as Kantian ethics, it is important to understand the motivations behind specific forays into the unknown. This rationale is perhaps one of the most broadly used and essentially requests a deferral of decision-making not because the consequences are known, but because the consequences are unknown and that in itself suggests that there are other things at play that we just aren’t able to consider. For example, if were to think about the consequences and problems that would quickly arise with the successful cloning of a human being, what would be the negatives? What would be the positives? Our world is quickly approaching that kind of future and sometimes we regulate because we want to prevent that future from occurring until we have a better understanding of what the externalities will be of that decision. Although there is no federal U.S law on human reproductive cloning, many states and international organizations have laws that explicitly ban it from taking place.
What is the difference between a law and a regulation?
Laws are created by the legislature, debated, passed through the house of representatives, the senate, and then the executive to be signed into law. A regulation is created by a government agency (FDA, SEC, FBI) and implements a given law. For a regulation to pass, the government agency holds a public hearing, goes through a notice and comment period, and then makes a decision after which it becomes a regulation.
Downsides of Regulation
Applying regulation to these systems could also hurt. Regulating can also be thought of a subsidy to a particular industry. Regulation can be thought of as a form of “insurance” provided by the government that protects and insurance that is paid for by society.
Those who oppose regulation of technology by the federal government or government at any level worry that regulation will stifle the rapid economic and value growth that technology creates. Technology has become the cornerstone of the American economy in many ways and by imposing restrictions that affect core internet business models or requiring reporting that dulls any competitive edge. Historically, private companies argue that reporting and standards adherence cause increase in operating costs because of the additional burden it puts on the business.
Regulation can also increase the power and influence of government in the daily lives of Americans. Since the beginning of America, people have been wary of centralizing power to a single force that can move unilaterally and enforce their will. Tyrannical power and centralized overstep seems to be the biggest fear of regulation being imposed. Although we have judicial courts that determine the current application and extension of the law, many Americans fear of government forcing private companies and individuals to do things that go against their best interest or their ability to self choose their future. This idea of self determinism is the strongest argument against regulation, but it must also protect ideals of justice in which one’s self determined future goes against or violates someone elses.
Means by which Regulation Comes About
Now that we understand why we regulate, we consider the ways in which we regulate. There are three main institutional mechanisms by which we are able to regulate: agencies, legislature, and international organizations.
Administrative agencies are the first line of defense in creating legislation. Agencies are created by congress but run by the executive. When laws are passed by congress, they don’t want to get into the specific details of the laws. Agencies have scientists that understand the specifics of a problem and thus have more domain expertise about how to go about determining the specifics of how to regulate. In addition, they are external to the lawmakers. When the way in which the law is implemented is controversial, legislatures can point their finger to the agencies as being responsible. Administrative agencies have gained a lot of power recently because of their domain expertise.
Regulation can also be created by the legislature. Regulation by the legislature has become problematic recently though. While the legislature was created with the specific purpose of creating law, it has become locked in a deadlock between parties. Parties are able to veto each others legislation which leads to “vetogates” where no legislation is ever passed, let alone complex legislation.
The legislature is still able to include the legislative process of the administrative agencies though money and investigatory powers. The legislature controls how the funding sources for administrative agencies through appropriations committees, which are able to exert their control. The legislature is also able to subenia . Congress can demand, under penalty of threat of arrest and detention (either themselves (which last occurred in 1935) or through the Department of Justice). A subpoena is a legally enforceable demand for documents, witness testimony. The Supreme Court has ruled that Congress needs the ability to subpoena in order to be able to investigate. As such, Congress’s powers to subpoena are not unlimited. Congress only makes subpoenas for the purpose of making laws and not for the purposes of enforcement or to expose wrongdoing.
International organizations have the power to create regulations. While in the United States we don’t typically think of them, internationally they are more helpful for establishing standards and norms. While they lack formal authority and enforcement, for external reasons, many countries don’t want to be viewed as a country that is trying to break international precedent. For example, if a country is not obeying ways in which to properly dispose of nuclear waste, it could be viewed by other countries as trying to create a nuclear weapon which could have consequences.
Companies have the knowledge but often lack the trust for creating regulation because frequently they fail to take into account the needs and wants of the general public. Their priorities can leave them with blindspots to the concerns of other parties, and too much collaboration between companies could turn into antitrust issues that fundamentally harm the American citizen. Civil society is innovative, unconstrained, and can generally shape the norms that it follows, but it fundamentally lacks an official role or an ability to enforce these norms. Courts, at least in common law countries, are the key backstop to enforce the law and promote integrity. However, their separation from the industries that they try to regulate leaves them with expertise issues and a general lack of knowledge, and often they must work with limited information to come up with solutions that affect entire industries.
All this is to say that there is no perfect institutional mechanism for regulating industries, and most solutions require the collaboration between different organizations.
Upon identifying how the regulation will be agreed upon, enacted, and enforced, it is important to also consider the way the law will cross cut many spectrums such as the regulation’s flexibility, focus on outcome, normativity in society, and how it will evolve over time.
Rules Versus Standards
In law, there is a difference between rules and standards. Rules are more clear and rigid, while standards are less constraining than rules. While rules might define specific conditions and consequences, standards might only give options and considerations. For example, a rule might establish having “no drones in the park,” while a standard might be phrased as “no dangerous use of drones in the park.” The rule of no drone usage is clear, but what constitutes “dangerous use of drones”? For regulation, institutions must decide whether to maximize clarity or to impose standards to rule on a case-by-case basis.
Substance vs. Process.
The question of whether the agency must choose a safe and reasonable policy vs the agency must develop a thorough record and answer every legitimate criticism of its position. When an agency proposes some level of regulation or not, does it act in a sensible way? Has the agency done the right thing? For example, in the Vermont Yankee Nuclear Power Corp. v. Natural Resources Defense Council, Inc. case, the United States Supreme Court held that a court cannot impose rulemaking procedures on a federal government agency.
Formal law vs. Norms/guidelines.
For example, should letting semi-autonomous agents roam the Internet be prohibited by law, or should social sanctions and institutional policies discourage such use? Which one would be more effective? Which one would be easier to implement?
Adaptation vs. Change in doctrine.
When new technologies are created, how should the law adapt? Take the case of Intel vs. Hamidi, where the California Supreme Court held that a former Intel Corporation employee’s emails to current Intel employees, despite requests by Intel to stop sending messages, did not constitute trespass of Intel’s system or trespass to chattels. The trespass to chattels is a tort whereby the infringing party has intentionally interfered with another person’s lawful possession of movable personal property. The Supreme Court declined to extend common law trespass claims to the computer context, absent actual damage.
Cost Benefit Analysis
While we may not be able to entirely resolve the above themes, how do we justify regulatory intervention and how do we specifically go about making these judgement calls. There are a few methodological frameworks which are used to regulate. For example, there are technical bounced process where we try and quantify the costs and benefits of a proposed regulation. If the benefits exceeds the cost then the regulation is considered good and passes.
When making a cost benefit analysis, there are typically three assumptions that are made. FIrstly, that everything can be modeled. The more that can be modeled in the system will result in a more accurate model. The second assumption is that the aggregate of everyone is more important than that of the individual. Regulations look at benefit to society as a whole. It is not concerned if certain people get more or less, what is important is the overall effect on society. Thirdly, a future benefit is worth less than an immediate benefit.
At the heart of the cost benefit analysis is what social scientists call Kaldor-Hicks which focuses on the aggregate and not the individual well being to society. If you apply Kaldor-Hicks across every issue, you might experience a backlash in a way that you just don’t expect between of the initiate interactions. Kaldor-Hicks can also make simplified assumptions about people’s preferences and how they can be modeled. You can’t model everything and appease everyone. If you did that with everything there could be one group that continues to be marginalized which can lead to unexpected political consequences. For example, while quantifying people’s money might be easier, quantify health, and people’s preferences for leaving the world a better place for their children can be harder to model. We can become disationed that everything can be trackable and we can reduce the complexity of making complicated decisions, when in fact these are not possible. People’s preferences change over time. How have your preferences changed from when you were a kid, to when you were in high school, to when you were an adult? Lastly, they can oversimplify a complex problem. We might focus on optimizing for one issue but in fact the issue might be intractably hard. We will be gaining precision at the expense of getting to the frankness of the situation.
Case Study: Cost Benefit Analysis of a Change in Air Pollutant Standards for Industrial/Commercial Boilers
In 2012, the EPA was deciding the impact of new regulation that would limit the emission of carbon monoxide, hydrochloric acid, and mercury from commercial boilers and used cost benefit analysis to help make their decision and their case as to the reason why the regulation made sense. The EPA eventually estimated that it would cost $2.2B in additional costs per year which includes capital costs to build newer systems, the maintenance of those systems, testing/monitoring the compliance of standards, and actually operating those new systems. That sounds like a tall bill, undoubtedly, but they estimated that the benefits would be almost 13–29 times worth the $2.2B. This benefit includes things like the health benefits from reduced exposure to particulate matter, the small net increase in jobs in some sectors, and a variety of other factors. Of course, the companies operating these boilers found numbers that strongly disagreed with this analysis because a majority of those benefits would not be reflected on their bottom line, but eventually gave way to the eventual regulation.
Cost-benefit analysis had many pros such as being able to quantify and model an incredibly large and complicated system so that legislators could better understand the impact of their regulation, but one of the largest downsides was that it was hard to do a historical analysis at whether the early estimates were actually accurate or lived up to their claims. The government, oftentimes, does not have to prove after the fact how accurate the cost-benefit analyses were meaning that we have very little insight into if the regulation did what it said it was supposed to do.
Some of the most common concerns with cost benefit analysis is the questions surrounding social equity. Simply tabulating up the costs and benefits of any regulation might fatally ignore concepts of social welfare and distributive justice. For example, although the emission of potentially harmful particulate matter might affect each human the same, the cost benefit analysis might not take into consideration the vulnerability of the population that is being impacted. Often seen in cases of “NIMBY” (Not in My Backyard) movements, the potentially harmful or undesired development/operation will move from wealthy neighborhoods to poorer ones. Although both the wealthy and poor might roughly be susceptible to sickness from the particulate matter, the wealthy might have the means to pay for treatment while the poor might not. This greatly underestimates the costs of projects like the one above and disproportionately affects vulnerable populations such as the poor or minority communities.
One other major concern and perhaps the largest is the accuracy and the precision of the cost benefit analyses. Often conducted by federal agencies and only supported by sparse guidelines from the early 1990s and the 2000s, cost benefit analyses come under heavy scrutiny and criticism because of the lack of standardization and the ability to “fudge” the numbers as necessary.
R-MUSIC: A Framework for Understanding When Regulation Is Needed
To overcome the issues with the cost benefit analysis, it could be better to take a more holistic approach to regulation. The first thing to consider is the reasons to regulate. There may be intertemporal utility conflicts, non-competitive markets, information asymmetries, or the need to maintain institutions for societal decision-making.
At the core, considering the reasons to regulate or the motivations behind ones’ actions call into question individuals’ implicit theory of the world. At the core of Kantian ethics and the seed for many other ethical systems, considering the motivations will reveal key metrics that define success of the regulation. For example, in the consideration of the reasons to regulate, a decisionmaker may call into question rational expectations or behavioral assumptions. Rational expectations theory is the idea that current expectations in an economy or system are equivalent or will hold true to what the future state of the economy will be. Behavior assumptions, for example, are behaviors acquired through learning and subject to social influence meaning that the reached value judgements differs from person to person based on experience. Zooming out further, decision-makers must decide if they want to optimize or “satisfice” (an amalgam of satisfy and suffice) meaning whether to pursue the optimal solution or one that satisfies the minimum satisfactory condition. If they do decide to pursue an optimal solution, another factor to consider is whether to follow the Pareto principle meaning that at least one party benefits and nobody is made worse off or Kaldor Hicks efficiency, a more utilitarian standard, in which the net gain to society is distributed in such a way that even potential losers can be compensated from the net gain. By considering the primary motivations and justifications behind the regulation, the conversations that arise will give regulators and the public transparent understanding of where each decision is coming from and what it hopes to contribute to the body politic.
The second thing to consider is the mechanism by which the regulate will be implemented. As we have been discussing at length throughout the writings, there are many ways for regulation to be observed in society and the question of who makes the decision is pivotal. Many of the conflicts we have seen since the foundation of American government is the extent to which the decision-maker will be the individual, directly elected local officials, or to national legislative or administrative bodies. The question of who makes the decisions impacts the interpretation, implementation, and enforcement of the law at varying scales. If the mechanism is a centralized one such as a federal agency, then perhaps the interpretation will be more strict to the letter of the law and its implementation far-reaching and a difficulty in its enforcement across the large nation. If the mechanism is more distributed such as at the local level, perhaps the interpretation may be more loosely held and the implementation very granular with strict enforcement. The policymakers must find the mechanism to regulate that considers these three spectrums and most optimally fits for what their aforementioned motivations are.
The third thing to consider is the use to which the mechanism is being put forth. After months of Congressional hearings, back and forth between interest groups, and careful negotiations with political members, it is often easy to lose sight of what the use of the regulation is for. This consideration helps decision-makers establish an ideal, aspirational goal of what the regulation will help do. This consideration will set the standard of what this regulation, when enforced properly will help ultimately do and lead to for citizens. For example, in the above example involving the EPA regulation of particulate matter, the use of the mechanism might be to set standards on what clean air that citizens are afforded the right to. This third consideration helps understand whether this regulation will serve more as a rule or a standard or an aspirational goal of what the world should look like. For example, the legislature might pass rules about what companies can and cannot do in polluting the air and the EPA might define the threshold standard that companies cannot pollute above and the individual eco-conscious citizen might aspire that companies do not pollute, but all have varying scales of the use of the mechanism.
Institutional considerations are ones that understand the decision in the context of larger institutional capacities, precedent, feasibility, and the trade-offs relative to alternative plans of actions presented by different stakeholders. This final consideration helps decision makers convene and participate in democratic deliberation that shapes the final ideas of the regulation and its implementation that consider societal welfare and the opinions and voices of a diverse set of stakeholders ranging from those who will implement, those who will feel the impact, and those who will benefit/harm from the decision. This consideration will appear more like a cost benefit analysis, but rather than just focusing on the quantitative results of the set of considerations, it will place a heavy emphasis on the qualitative feedback of the stakeholders in the decision. For example, in the aforementioned EPA example, we might consider whether or not the EPA has the human capital resources to be able to manage and maintain strict regulations on this industry among the many other tasks they have to accomplish? What kinds of people will be affected by the regulation and to what extent will this decision impact them versus other populations of different economic, ethnic, and geographical background?
In the next two case studies, we will examine cases where the questions and framework of R-MUSIc could provide helpful, especially because they deal with pressing and emergent issues of artificial intelligence. These case studies offer insight into how the situations that technology present depart from the common legal scholarship and how it necessitates a reframing of the way our legal system answers questions. The problems facing the legal system is not necessarily that it is unequipped to answers questions of emerging technology and the challenges they face, but that we must fully appreciate and contextualize the uniqueness of these situations. Fortunately the common law system is built in such a way that we can learn from cases such as the ones below to help inform new paradigms of thinking, such as R-MUSIc.
Case Study: Torres v. North American Van Lines, Inc
In Torres v. North American Van Lines, Inc., a truck driver was killed in an accident because the truck driver didn’t get enough sleep, and the jury awarded punitive damages to the family because they determined that there was gross negligence on behalf of the company.
In relevance to AI systems, part of the trial evidence included a discussion of “the company’s computerized data processing system,” which prevented drivers from driving for too long. The court argued that if the company has the technology to understand the risk and keep track of safety, then they are negligent because they had the opportunity or the technical means to prevent this accident from happening. Does the company have the ability to track driver safety, and if they do, why didn’t they use it? Are they liable for not using information they had, but did not know how to use properly? Even more so, with the advancements in machine learning, the company would be able to calculate overall risks in their driver safety with a multitude of other factors.
Cases like these may both encourage and prevent automation. For example, the court said that even if a company might have a person in the office that is in charge of keeping track of driver working time, if it has software that is more accurate than people, then the company would still be negligible because it didn’t use the software that was already available to them. This may cut against automation by having uncertainty about the benefits and what constitutes a “reasonable person.” It may also encourage automation through the submission of evidence to support gross negligence but also by redefining the “reasonable person” standard. Bottom line: Can tort law take into account the factors that cut against automation (erosion of knowledge in organizations, systemic failure, lack of security) when making decisions? Torres v North American Van Lines, Inc. sets an interesting precedent because the courts were essentially finding the defendant at fault for not using computer systems and data points they had to stay compliant with safety standards. Although the tort system awarded $2,500,000 in damages to the Torres family, with the inclusion of simple data modeling and verification systems, the defendant might not have been in this situation. Surely, the cost of digital transformation is large and requires a large upfront investment, but as this case shows: the cost of not adapting to available technology and properly using the information to fuel compliance and business operations could be a significant risk to the business.
Case Study: Houston Federation of Teachers, Local 2415, ET AL. vs Houston Independent School District
In 2010, the Houston Independent School District implemented a new teacher evaluation system in hopes of “having an effective teacher in every classroom.” In the name of transforming into an organization that proves quantitative impact on the students’ performance, the new software implemented a proprietary algorithm called the Educational Value-Added Assessment System (EVAAS). The EVAAS system tried to track the teacher’s impact on student test scores. Even the court readily admitted the algorithm was much more complex than they had the ability to understand, but they argued that the measurement was a proprietary abstraction of several data points. The teacher union found this new “data-driven” approach as a threat to their due process because in this new system they were able to be fired for not hitting “satisfactory” numbers. They accused the school district of depriving them of their 14th amendment right that protects them against unfair or mistaken deprivations of life, liberty, or property. The crux of the difficulty in the case was that they had to balance the secrecy in the algorithm and the right for teachers to understand the system’s process.
One major concern, however, was that even though the EVAAS metric was being used in the assessment of the job performance of the teachers, even the HISD administrators had no way of knowing how the algorithm worked, if the data was correct, and how specific variables impacted the final outcome (i.e the weights behind certain variables). The teachers union argued that not only did the appraisal system violate their right to due process, but it was unfairly biased against specific teachers of low socioeconomic students and those for whom English is a second language. They presented compelling evidence that test scores for those populations is typically lower and that if the algorithm didn’t properly account for this, an excellent teacher in a challenging classroom might be fired.
This court case, having been argued in the United States District Court in the Southern District of Texas, presented a matter of first impression for this specific court. A matter of first impression is when a court case is presented to the judge that presents a novel question or issue for legal interpretation that they may have never seen before. This often happens when it comes to cases of newly passed legislation or recent cultural/technological advances that require the consideration of decisions from other courts, commentaries by legal scholars, and the arguments made by the lawyers in the case. These cases of first impression are equally exciting and dangerous for the common law system because not only do they push legal interpretation forward for a variety of emerging situations, but because it tries to answer questions that have yet to be certain to have answers or that the legal system is equipped to handle.
The questions about due process when it comes to algorithms is asked not just in cases like this by the teachers’ union in Houston, Texas, but in cases regarding criminal justice in Wisconsin to intellectual property in Delaware. Perhaps some of the most interesting legal scholarship to be undertaken by those who understand cross cutting themes of computer science and the legal system lies in this intersection of how due process, equal rights protection, and more fit into the inherent abstractions that are algorithms like the ones that the Houston Independent School District employed this decade. The common law system is constantly adapting to the legal challenges that arise in an ever changing world, but it also necessitates those who fundamentally understand the technology to legislate, regulate, and enforce them.
