“Ensuring Safe, Secure, and Trustworthy AI”: What those seven companies avoided committing to
On July 21, 2023, in a photo op featuring only men in suits, President Biden announced that seven companies (Amazon, Anthropic, Google, Inflection, Meta, Microsoft and OpenAI) had agreed to voluntary commitments regarding “safe, secure and trustworthy AI”.
The text of the voluntary commitments is available (via a White House “fact sheet” describing this as a step in the Biden-Harris administration’s broader efforts around regulating AI).
I wouldn’t expect any voluntary commitments to add up to what is needed, especially with that list of actors, but it is still informative, I think, to look at the document, both in terms of what is missing from it and what is inappropriately included.
What’s missing
The biggest lacuna is data and model documentation. There is a body of work going back to at least 2017, representing convergent scholarship, including across many industry labs, finding that a crucial first step in mitigating harms from pattern matching systems is providing detailed documentation of the training data they are extracting the patterns from.
This scholarship doesn’t just point to the importance of documentation but also includes specific proposals for what to include in the documentation. These include Datasheets for Datasets (Gebru et al 2018, 2021; originally at Microsoft), Model Cards for Model Reporting (Mitchell et al 2019; Google), Factsheets (Arnold et al 2019; IBM), Data Statements for Natural Language Processing (Bender and Friedman 2018, Bender et al 2021), Nutritional Labels for Data and Models (Stoyanovich and Howe 2019), Data Nutrition Project (Holland et al 2018, Chmielinski et al 2022), and others.
In Stochastic Parrots (Bender, Gebru et al 2021, p.615), too, we stressed the importance of dataset documentation:
“Without documentation, one cannot try to understand training data characteristics in order to mitigate some of these attested issues or even unknown ones. The solution, we propose, is to budget for documentation as part of the planned costs of dataset creation, and only collect as much data as can be thoroughly documented within that budget.”
Documentation empowers us to ask questions, including:
- Procurers: Is this system appropriate for the users I anticipate will interact with it?
- Policymakers: Are rights respected in the development and deployment of systems?
- Community activists: What patterns are being reproduced which adversely affect my community?
Conversely, without access to dataset documentation, we are much more poorly positioned to proactively mitigate harms associated with automation.
“As consumers of datasets or products trained with them, NLP researchers, developers and the general public would be well advised to use systems only if there is access to the type of information we propose should be included in data statements.” (Bender and Friedman 2018)
The voluntary commitments document even says (under point 7) “Companies making this commitment recognize the importance of avoiding harmful biases from being propagated by, and discrimination enacted by, AI systems.” And yet it doesn’t say anything about the obvious first step to achieving this.
It’s not that the corporations don’t know about the importance of data and model documentation, nor the existence of methodologies for producing it. It’s that they are unwilling to commit to it voluntarily. (Recall the tantrum that OpenAI’s Sam Altman threw about the EU requiring documentation of copyrighted material used in model training.)
For the interests of the people to be protected, we must collectively, through government action, require data and model documentation.
What’s carefully carved out
The voluntary commitments do talk about the importance of watermarking and provenance:
“Companies making this commitment recognize that it is important for people to be able to understand when audio or visual content is AI-generated. To further this goal, they agree to develop robust mechanisms, including provenance and/or watermarking systems for audio or visual content created by any of their publicly available systems within scope introduced after the watermarking system is developed.”
But these commitments are as solid as swiss cheese, with two gaping holes carved out.
First, the commitments only address “audio and visual” content, with exactly zero mention of synthetic text. When OpenAI set up the easy interface to ChatGPT, when Meta briefly provided an interface to Galactica (misleadingly billed as way to access scientific knowledge), when Microsoft and Google incorporated chatbots into their search interfaces, they created the equivalent of an oil spill into our information ecosystem. Anyone can go at any time to one of these sources (except Galactica, which was taken down) and produce seemingly authoritative text in the style of their choice. They can then easily post this text on the internet where others might turn it up without knowing its origin.
We’ve seen previously respectable journalistic sources quietly posting synthetic text as if it were real reporting, chatbot output threatening to dilute the value of sites such as StackOverflow, someone letting a chatbot loose on reddit, a hapless lawyer trying to enter fake cases extruded by ChatGPT as part of a legal brief, and courts in Colombia, Mexico and Peru turning to ChatGPT for input in decisions, among other such synthetic media spills. (Not to mention the 1000s of journalists who thought it would be cute to have ChatGPT write part of their articles, only to reveal below the fold that it was fake.)
The reason I make the analogy to oil spills is that this isn’t just about the harms to the person who initially receives the information. There are systemic risks as well: the more polluted our information ecosystem becomes with synthetic text, the harder it will be to find trustworthy sources of information and the harder it will be to trust them when we’ve found them. Rich Felker makes this point well over on Mastodon in a thread on the importance of provenance, without which information is just words.
These risks (much like the climate crisis) are present and real. And yet: the voluntary commitments, while talking about the importance of provenance, leave out synthetic text entirely. And while yes, watermarking synthetic text (which can afterall be copied and pasted) is non-trivial, there are ideas for how to go about it, including Kirchenbauer et al’s proposal which just won an outstanding paper award at ICML 2023. And yes, even if some players commit to watermarks (meaning their synthetic text can be removed by anyone looking for clean information sources), there will still be others who don’t. But it still matters! Any pollution is bad, but more is worse. The fact that the Exxon Valdez spilled 11 million gallons of oil into Prince William Sound didn’t make the BP Horizon disaster (205 million gallons into the Gulf of Mexico) any less terrible.
Carve-out number one was text not included. Carve-out number two: the voluntary commitments don’t apply to current models:
“Scope: Where commitments mention particular models, they apply only to generative models that are overall more powerful than the current industry frontier (e.g. models that are overall more powerful than any currently released models, including GPT-4, Claude 2, PaLM 2, Titan and, in the case of image generation, DALL-E 2).”
So yeah — this is weak sauce. The companies say they’ll try to mitigate some pollution down the road, but do not wish to do anything about the toxic waste they’re currently spewing.
What’s weirdly included
It’s also worth noting some things in the voluntary commitments that are beside the point at best. The final item of the voluntary commitments reads as follows:
8) Develop and deploy frontier AI systems to help address society’s greatest challenges
Companies making this commitment agree to support research and development of frontier AI systems that can help meet society’s greatest challenges, such as climate change mitigation and adaptation, early cancer detection and prevention, and combating cyber threats. Companies also commit to supporting initiatives that foster the education and training of students and workers to prosper from the benefits of AI, and to helping citizens understand the nature, capabilities, limitations, and impact of the technology.
This isn’t about constraining companies from bad behavior and in fact reads like all “AI for social good” initiatives: It’s irrelevant whether these companies believe or want us to believe they are working on using “AI” to combat climate change. That doesn’t obviate the harms they are doing (including harms to the climate, through profligate use of water and energy resources). This is fluff, it’s PR, and its inclusion in this document just underscores how it the commitments are really an exercise in marketing and an attempt to forestall meaningful regulation. We pinky promise to be good, now please go away while we continue to practice massive data theft while creating poorly engineered “everything machines” that can’t possibly be evaluated.
Perhaps even more alarming though, is two of the points included under “Safety”, where the companies voluntarily commit to “red-teaming” to test for various risks, including:
The effects of system interaction and tool use, including the capacity to control physical systems
The capacity for models to make copies of themselves or “self-replicate”
Where do you hear people going about those kinds of risks? It’s the AI doomers going around telling fantastical horror stories about “existential risk”. Of course, given the presence of OpenAI, Anthropic and Inflection (all deeply influenced and in fact funded by actors in the TESCREAL space), it would almost be more surprising not to see any traces of these ideologies in this document. Still, this document presumably had input from the White House, not just the companies (one hopes!) and it’s distressing that the White House wasn’t able to catch and filter this bit of nonsense.
What’s next
Here’s what I’d like to see:
- Policy makers talking to people with expertise in the real (current and immediately foreseeable) harms of so-called “AI”. There should be far more input from such folks (scholars, community activists, journalists) than from the people leading the companies that ought to be regulated.
- Regulation that demands transparency, disclosure, accountability, privacy and data rights.
- Immediate action by federal agencies applying existing law (as the FTC is already doing) and incorporating best practices into federal procurement procedures (as Reps Lieu, Lofgren and Stevens have asked).
- Policy makers who don’t fall for the narrative that tech is moving too fast to be regulated. Regulation protects rights, and those endure.
… and, of course, continued engagement and discussion of what else is missing from these voluntary commitments and what more needs to be done.
