A Clear Guide to Understanding AI’s Risks: Part 4
What Are We Doing About These Risks?
This is part four of a 4-part essay. To access the other parts, use the links below:
II) Risk 1: The Misalignment Problem
IV) What Are We Doing About These Risks? (this is what you are reading)
IV) What Are We Doing About These Risks?
As of today, there is no concrete plan in place to deal with any of these risks.
Recent months have seen this conversation surface into the public sphere. There have also been some initial efforts to regulate this technology. But we are still far from instituting any real safeguards against these dangers.
In March 2023, the Future of Life Institute released an open letter calling for a pause in AI development:
AI systems with human-competitive intelligence can pose profound risks to society and humanity. [These] systems are becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.
[This pause should be used] to jointly develop and implement a set of shared safety protocols for advanced AI development. In parallel, policymakers [must] dramatically accelerate development of robust AI governance systems. These should include systems to help distinguish real from synthetic, liability for AI-caused harm, funding for technical AI safety research, and well-resourced institutions for coping with the dramatic economic and political disruptions (especially to democracy) that AI will cause.
This letter was signed by over 30,000 people, including notable figures such as Elon Musk, Stuart Russell, Steve Wozniak, Andrew Yang, and many others. It started a global discussion about these risks and also proposed some specific ways to start mitigating them.
Why did the letter ask for a pause?
This letter recommended pausing AI development for 6 months because we just don’t know how to deal with these risks yet.
Right now, humanity doesn’t have the tools to fix the problems we see on the horizon. We need to do much more research to determine which tools would work, or whether we can even find any tools to mitigate these harms.
Recent months have seen some solutions proposed, but we need to actually test these in real situations before we can confidently release powerful AI into our world. Some things that have been proposed recently include:
- Watermarking, to reduce the prominence of AI impersonating humans
- Red-teaming, to mitigate the risks of AI systems producing harmful output when released. Harmful output could include teaching people how to make bio-weapons, or providing access to dangerous information
- Provably safe AGI systems, to mitigate the risk of misalignment
But these solutions have not been tried at scale yet, and it is unlikely that they are the only tools that would be needed to safely release a superhuman intelligence into the world.
If AI ends up being good for us, then there’s no harm waiting months or years, we’ll get to the end point anyway…if it’s harmful, then we just bought ourselves extra time to strategize the best ways to respond and understand how to combat it.
James Grimmelmann, professor of digital and information law at Cornell University.
This open letter was published in March 2023. But since then, instead of slowing down, AI development has actually sped up. As you read this today, hundreds of AI companies across the world are competing against each other to develop and deploy this technology as quickly as possible. The firms developing this technology are not incentivized to focus on public safety; they are instead only focused on staying ahead of the competition and capturing as much market share as they can.
Moloch
Recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one — not even their creators — can understand, predict, or reliably control.
It would be reasonable to assume that these companies are not concerned about these risks. But in reality, the leaders of these companies are much more concerned about these harms than they state publicly, and in some cases, they are actually looking for a way to pause development themselves. But they are unable to do so due to the pressure they face internally.
Max Tegmark, President of the Future of Life Institute, spoke with them and shared what he learned with The Guardian and The Lex Fridman Podcast:
“I felt that privately a lot of corporate leaders I talked to wanted [a pause] but they were trapped in this race to the bottom against each other. So no company can pause alone…[they are] afraid of expressing their fear. There’s just so much commercial pressure. If any of [them] say that this is too risky and I want to pause, they’re going to get a lot of pressure from shareholders. If you pause, but the others don’t pause, you don’t want to get your lunch eaten. Shareholders even have the power to replace the executives in the worst case.”
Take a moment to appreciate this absurd situation. The people developing this technology are fully aware of the catastrophic risks it poses, and actually want to slow down, yet they cannot due to the pressures they face from the market. If Google’s CEO is concerned about the risks of AI, he cannot decide to simply put development on hold to focus on the risks, because that would mean Facebook and Microsoft would simply race ahead and steal their users, and he would be ousted by the shareholders in favor of someone who would continue building this technology.
This is not a small fringe of leaders either — 42% of CEOs that attended the recent Yale CEO Summit believe that AI can destroy humanity in the next five to ten years. Various leaders of AI firms in the US (including the CEO of OpenAI) also signed the following statement released by CAIS in May 2023:
Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
Our smartest minds recognize the dangers this technology poses, yet they continue to race forward in reckless development anyway.
This situation has occurred in human history before, and has a name. It is known as Moloch. Moloch describes a scenario where everyone knows they are making bad choices, but they cannot make any other choice given the incentives they face. This is not caused by a single person or entity, but simply due to the rules of the game.
The nuclear arms race in the Cold War occurred because both the US and the Soviet Union did not want to end up on the losing side of a potential war, so they ended up building thousands of nuclear warheads trying to outdo one another. But as they did this, it became clear to both of them that each new nuclear warhead they built was only going to lead to the same happening in the other country, and this eventually got to a point where there were so many nuclear warheads that any war between them would mean that both sides lost.
The incentives faced by AI firms today means that they will continue to deploy larger and more capable AI models in order to stay ahead of the competition, regardless of the risks.
This is the most impactful technology that humans will ever develop. Should we let it play out according to the forces of the free market?
This is a moment in time so pivotal, it’s deserving of reverence and reflection, not something to be reduced to a corporate rat race of who can score the most daily active users.
So, how do we escape this race to the bottom?
Side note: In addition to market forces, there is also a geo-political rivalry which is driving some of this AI race, as the US bids to outcompete China. But as I explain later, this is also not a valid reason to develop AI recklessly.
How to defeat Moloch
Persuade your fellow citizens it’s a good idea and pass a law. That’s what democracy is all about.
Antonin Scalia
The only way to defeat these incentives is to create an external force which supersedes the pressure these companies face to continue developing this technology. Since they cannot pause through any internal force, the general public and the government need to step in and take control of this situation. Currently, the forces faced by AI firms are pushing them in only one direction:
We have been able to defeat Moloch before, as described in Dazed:
In the 1980s, we were living under the shadow of the nuclear bomb for decades. Then, in 1983, a horrifying TV movie, The Day After, was screened to more than 100 million Americans (almost half the population at the time). According to ABC, the streets of New York emptied out as people gathered for mass viewings of the film, which dramatized a war that escalated into a full-blown nuclear exchange between the US and the USSR, and explored the devastating aftermath. Following The Day After, a panel of the country’s leading politicians and intellectuals featured in a televised debate on the pros and cons of nuclear proliferation. In 1987, US president Ronald Reagan — who wrote in his diary that The Day After was “very effective and left me greatly depressed” — met with Soviet Premier Mikhail Gorbachev to sign a treaty that resulted in the reduction of the nuclear arsenal on both sides. Another nuclear arms treaty was signed in 1991 which reduced the number of nuclear weapons in the world by over 80% in the following years.
Public awareness of the issue is the first step to defeating the forces that are driving this race forward.
Attempt 1 (May 2023 — Present)
Despite the open letter by the Future of Life Institute not leading to the pause that was hoped, it created an environment where it was more socially acceptable to take these risks seriously, and applied external pressure on the industry to slow down.
We released this letter because we want these tech executives to do what their heart tells them, by providing enough public pressure on the whole sector.
In the months following its publication, there were some promising actions taken by governments to regulate this technology.
In June, lawmakers in the EU approved a set of regulations to prevent AI’s use in many harmful applications. The US also took a first step in regulation recently when Joe Biden recently signed an executive order in October which aims to tackle various AI risks. Most recently, the UK held an AI safety summit this November which convened top leaders in the field to discuss how to mitigate some of these catastrophic scenarios. Some specific policies proposed by these regulations and summits include:
- Instructing federal agencies in the US to start creating standards for what constitutes a “safe” AI model
- Instructing AI companies to share the results of their safety tests with the US government
- Preventing the use of AI in harmful applications such as social scoring and harmful policing in the EU
- The creation of an independent organization which releases an annual report on the “state of AI”, similar to the IPCC for climate change
These policies represent a good step forward in the fight against AI risks and against Moloch. They also show that global leaders are starting to take these risks more seriously, which is necessary. Overall, these were very positive and somewhat surprising events which could not have been possible without the recent public outcry about these risks.
However, in terms of having a serious impact on AI development, there is still a large gap in these regulations. The policies proposed so far do not place any concrete, enforceable guardrails on this technology’s deployment. Helen Toner, Director of Strategy at Center for Security and Emerging Technology (CSET), states, “The Executive Order only requires companies to inform the government of the results of the safety tests. It is totally unclear what would happen if a company were to [fail the safety tests]. This is really putting the ball in Congress’s court to reinforce the things that the executive branch just can’t handle on their own.”
Executive orders do not have the power to do much on their own — this was signed by Joe Biden because the US Congress is still far from passing any major AI legislation on their own. Most of this executive order is just meant for federal agencies to start taking a closer look at how this technology could be safely developed, rather than actually enforcing any restrictions for AI labs.
This is an important first step in the direction we want to go, but in order to actually affect AI development, Congress needs to step in, as stated by Biden himself: “This executive order represents bold action, but we still need Congress to act”.
The EU Act has many important facets as well, but it does not seriously protect us against any serious misalignment, economic, or weaponization risks yet. It also will not come into effect until 2025. This technology is changing every week, but the pace we are taking to regulate it does not take this into account.
If you were an AI lab today, there is actually not much that has changed for you after these recent regulations. To the public, it seems like there is a lot more discussion about these risks today, but discussion alone is not going to change the development of these AI models. We have formed countless committees and held multiple conferences over the past few decades to discuss how to combat the threat of climate change, but to this day, very little concrete action has been taken to mitigate its harms. We may not have decades to deal with the threat from AI — we need to act quickly and decisively.
The country with the largest and most impactful AI labs is the US, and the only legislative body which can actually have an impact on them is Congress. Senate Majority Leader Chuck Schumer has been bringing together experts to recommend policies to regulate this technology. Max Tegmark attended some of these meetings, and shared what he observed:
“It’s great that they’re actually having these conversations, but it’s not enough to talk the talk — they need to walk the walk also and actually pass some laws now. [In this meeting], I tried to raise the topic of the reckless race toward superintelligence, but another attendee [shut me down]. I really felt like I was in the movie Don’t Look Up.”
“I suspect that because AI’s implications are so vast and still so poorly understood [by members of Congress], that what we’ll ultimately end up doing is tracking more toward incremental, narrow fixes.” — Divyansh Kaushik, Director for Emerging Technologies at the Federation of American Scientists
As it stands, the odds of Congress passing a set of comprehensive policies to combat AI risks within the next few months are not high.
Our current situation still vastly favors the continued reckless development of this technology through the forces of the free market. In short, Moloch is still winning.
In the six months since the pause letter, there has been a lot of talk, and lots of photo opportunities, but not enough action. No major tech company has committed to transparency into the data they use to train their models, nor to revealing enough about their architectures to others to mitigate risks. Nobody has found a way to keep large language models from making stuff up, nobody has found a way to guarantee that they will behave ethically. Bad actors are starting to exploit them. I remain just as concerned now as I was then, if not more so.
Dr Gary Marcus, Professor of Psychology and Neural Science, NYU
Attempt 2 (Present — Future)
Recent regulations and proposals have been our first attempt to deal with the monstrous risks of AI. International consensus is growing on the dangers of this technology, and the frameworks that are needed for serious regulation are being put in place.
But we are still losing the fight against Moloch. We need to apply much more pressure to slow this race down. This can only come from concrete, enforceable government policies targeting AI labs. And these policies can only be enacted if the public creates enough pressure for them to be enacted.
This is where you come in. Right now, there are a growing, but still small number of people who are concerned about the serious risks of AI. We are still having too many conversations about how AI can write students’ essays for them, but not enough conversations about how this technology can cause the next financial crash.
To make change happen, we need people to focus on the right problems, and we need to increase the number of people who are aware of these problems. The most effective thing you can do in the fight against Moloch is to spread the word about these risks to as many people as we can.
Talk to your family. Talk to your friends. Talk to your coworkers. Attend a protest. If you are a powerful person in an organization, use your resources to spread the word about this issue. The policymakers who can actually take action on these issues are much more likely to act if everyone wants them to act. We were able to decrease the number of nuclear weapons in the world by 80% because the entire country united around the goal of not being annihilated in the next few years. We need to similarly form a critical mass of people which makes this issue impossible to ignore — currently, there are still too few people pushing for change.
The policymakers know what to do, they just are not incentivized to act yet. The Future of Life Institute posted a set of recommended policies for the US to guide the discussions that are currently underway. Some of the most important ones include:
- We need more research funding to help determine potential solutions to these problems
- We need to require AI developers to prove to third party auditors that their systems are safe
- We need to establish a centralized federal authority responsible for monitoring, evaluating, and regulating general-purpose AI systems
“Right now, there’s 99 very smart people trying to make [AI] better and one very smart person trying to figure out how to stop it from taking over. And maybe you want to be more balanced.” — Geoffrey Hinton
It is not just the US that needs to act. We need international, coordinated action on this issue. We must put political differences aside to confront an issue which does not discriminate based on country.
We need to start today. We are the generation that is shaping the most powerful technology our species will ever create. There is no time to waste.
Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks.
Stephen Hawking
A note on China
“Here in the US, you often hear people arguing against regulations by saying, ‘Well, if we slow down, China’s going to surge ahead.’ But I feel like that might actually be a false narrative.” — Russell Wald, Director for Policy at Stanford’s Institute for Human-Centered AI
Despite popular belief, China is actually much more concerned about the potential of AI runaway risks than the US is. It has already taken the lead on regulating this industry, and is taking large steps to monitor developments in this field closely.
“AI developers in China already face a far more stringent and limiting political, regulatory, and economic environment than do their U.S. counterparts.“ — The Illusion of China’s Prowess
The government in China would not want to lose control over their society through any means, and as we have learned, advanced AI systems pose the greatest risk of all.
Jeffrey Ding, an assistant professor of political science at George Washington University, states, “[Assuming that] China that will push ahead without any regulations might be a flawed conception…China could take an even slower approach [than the US] to developing AI, just because the government is so concerned about having secure and controllable technology.”
We cannot afford another “Moloch-style” war with another nation for a technological arms race. The only winner in a US-China AI race will be AI.
