Misplaced concerns: real risks of chatbots
There is a significant amount of concern regarding the potential misuse of chatbots, such as ChatGPT, Bing’s chat, Claude, and Bard, by bad actors. For instance, The New York Times published an article on February 8, 2023, titled “Disinformation Researchers Raise Alarms About A.I. Chatbot.” My argument, however, is that these tools may not be as useful for nefarious purposes as we might think. Instead, we should be more concerned about open-source language models (LLMs) and those developed by less responsible companies outside of US jurisdiction.
Image Generative AI
In the period between the release of DALL-E-2 and ChatGPT (which now seems like a separate epoch), the media expressed significant fear about the potential of recent image generative AI. This fear intensified when StabilityAI released Stable Diffusion, prompting headlines like “This startup is setting a DALL-E 2-like AI free, consequences be damned.”
Soon after, on September 7th, The Atlantic reported that “in short order, a subset of Stable Diffusion users generated loads of deepfake-style images of nude celebrities, resulting in Reddit banning multiple NSFW Stable Diffusion communities.”.
Yet, despite these alarming stories, I could not find any news articles about harm done to specific individuals through the generation of convincing fake pictures using Stable Diffusion or other leading generative image AIs. Deepfake porn and scams involving voice impersonation remain issues, but these problems existed even before DALL-E-2. Eliot Higgins’ photo story about Trump’s arrest, which was a political satire, should not be counted as evidence of misuse as it was clear they are fake. The truth is that generative AI has only democratized access to technology that was previously exclusive to professionals.
Misinformation
Returning to the topic of text-generating AI, there is significant concern about the hallucinations these AI systems create. However, their tendency to guess information they do not know in order to fill in gaps may not be useful for any applications, including misinformation campaigns. A person who sporadically utters false information is often perceived as a fool rather than a skilled liar. Voters are more likely to believe a notorious lying politician when their falsehoods align with existing prejudices, fears, or grudges. Successful disinformation campaigns typically present a coherent picture. For example, according U.S. Department of State, Russian propaganda often contains narratives such as
- “Russia is an Innocent Victim”
- “The Collapse of Western Civilization is Imminent”
- “Popular Movements are U.S.-sponsored ‘Color Revolutions’”
To support these narratives, an AI must consistently adhere to them without compromising itself by spouting random nonsense.
Generating fake news
There are concerns that LLMs could be used to generate articles that reinforce these narratives. David Rozado demonstrates in his post that he was able to turn GPT-3 into a highly politically biased model. In fact, one does not need to adjust the model that is available only for research and open source. It is currently possible to use GPT-4 to generate arguments for any position. For example, here is an article generated by GPT-4 about why internet forums should be banned
In fact, internet forums and the internet in general seem to have a much greater potential for causing harm than chatbots. No single entity has control over them. On the other hand, if generating misinformation via chatbots becomes a genuine problem, the chatbot’s owner can take measures to curb such usage. Requests or generated content that follow common misinformation narratives could be detected and filtered by the chatbot. Even if the filter is not perfect for every query, a limit could be imposed, such as allowing only 10 suspicious queries per week for each paying account. If this limit proves too restrictive for some legitimate institutions, it could be increased or removed on an individual basis. Granting individual exemptions from general policy would not even require new functionality, as restrictions should differ for various applications. Taking recent cooperation with OpenAI as an example: topics and situations in Role Play for Duolingo should be restricted, while Be My Eye which describes the photo of current surroundings of a blind or low-vision person should not censor what they might hear.
Alternatively OpenAI or other responsible provider could store suspicious results and let social media check if linked article or user profile isn’t generated through some trusted intermediary.
Alternatively, OpenAI or other responsible providers could store suspicious results and allow social media platforms to check if linked articles or user profiles are AI generated through trusted intermediaries.
While OpenAI’s tool for detecting generated text has limited accuracy, its effectiveness will increase when social media platforms apply it to all posts from a user, making it much easier to determine if they are a bot. Consequently, it seems much simpler for propagandists to use leaked LLaMa 65B parameter models from Meta or other LLMs and fine-tune them. This way, they could generate any number of articles while bypassing US regulations.
SMS/mail scam
Language models (LLMs) are already being used for scams. I have personally encountered two instances of SMS messages that began with a “Thai woman” mistakenly texting my number as if it belonged to someone else, and then introducing herself. A mini-Turing test I conducted revealed that the bot’s common sense level was certainly below that of ChatGPT.
The bot “knew” that one needs to go to space to fly a spaceship but did not find it suspicious that an interstellar passenger could speak decent English. A human scam operator took over once I sent a picture of myself (as an alien).
These scams are likely to improve, but I don’t think scammers would use OpenAI. This would require mass usage and could be easily detected, even if individual messages might not be. Open-source models seem like a much safer option. Moreover, current methods for detecting spam and scam SMS messages seem to work well, at least for my phone number. I receive only about one such message every two months.
Summary
In summary, concerns about the misuse of chatbots and AI-generated content are valid, but the potential for harm may not be as significant as it seems. Open-source language models and those developed by less responsible companies outside of US jurisdiction pose greater risks. Misinformation and disinformation campaigns, as well as scams using AI, should be addressed by developing more effective detection methods and implementing usage restrictions where necessary. It is crucial to strike a balance between ensuring the responsible use of AI and maintaining the freedom and benefits it provides to users.
