Can AI Do That? The Challenges, Limitations, and Opportunities of Generative AI
There is a lot of discussion in the public sphere about how tools that use artificial intelligence (AI) to generate images and text will “disrupt” or “revolutionize” various industries — from journalism to advertising to education. There are also a lot of news stories detailing specific opportunities, challenges and limitations related to these tools. However, I’ve found few online resources that provide a relatively concise overview of those opportunities, challenges and limitations. So that’s what I’ll try to do here.
First off, when I talk about “generative AI” or “AI content generators” in this piece, I’m talking about two things. First, there are text generators, like ChatGPT, which draw on large text databases and use AI to answer questions and prompts from users. Second, there are image generators, which draw on large databases of photos, illustrations and other graphics to create images (still images or videos) based on prompts from users. Odds are good that you already have some familiarity with these tools.
Now that we’re all working on the same definition of these tools, let’s look at some of the challenges they’re facing.
Legal And Ethical Issues
There are more than half a dozen lawsuits against the makers of AI text and image generation tools.
These range from a suit filed by the New York Times against OpenAI’s text generators in December 2023 to a suit filed by Getty Images against Stability AI in early 2023. In all of these cases, plaintiffs are arguing that companies are using intellectual property — from books and news stories to photographs and related metadata — to train their AI programs, and that they have done so without gaining permission or providing compensation to the people or institutions that own the relevant intellectual property. However, there are also more nuanced claims that vary from case to case.
For example, Getty Images argues that people who use Stability AI’s image generators are often getting manufactured images that are “bizarre and grotesque” — and that are also watermarked as coming from Getty Images. This, they argue, hurts Getty’s brand. In short, they don’t like the idea of Getty being associated with an image that purports to show, say, Queen Elizabeth II riding a unicorn with eight fingers on each hand.
While it is possible that AI companies may be able to reach financial settlements with plaintiffs in some of these cases — particularly those brought by corporations — there are also cases in which the plaintiffs are opposed to the fundamental idea of their creations being used to automate the creative process.
In other words, at least some of these cases will be fought to the bitter end.
These cases are relevant to potential users for at least a couple reasons.
First, it raises the possibility that any tool you spend time, money, or effort becoming proficient in could be pulled from use at the drop of a hat.
But it also raises the question of potential legal liability for people and institutions that use content generation tools.
For example, if the material produced by a text generator effectively reproduces copyrighted content written by someone else, that could raise some interesting legal questions. The same questions exist for images produced by AI programs, if the resulting images effectively reproduce images that are copyrighted or trademarked — not to mention legal issues that come into play if the image generator reproduces the likeness of a celebrity or other public figure.
What’s more, there may also be legal liability for users if they disseminate inaccurate or misleading information — even if the misinformation stems from mistakes made by generative AI. This has already happened at least once, with Air Canada being held responsible for incorrect information being shared with a customer by an AI-powered chatbot.
Lastly, a federal district court has ruled that users cannot copyright any content (images or text) created by generative AI. The court found that only works created by a human can be copyrighted.
In short, the use of generative AI tools may have significant legal ramifications for users, depending on how the users plan to utilize generative AI tools.
It is also important to note that, regardless of these legal issues, the points being raised by authors, reporters, publishers, photographers, and artists raise serious questions about the ethics of using content generation tools that make use of creative work without compensation.
In other words, even if you can do it, is it ethically acceptable to do it?
Now, setting aside the legal and ethical questions related to AI content generators, to what extent are these tools actually useful?
Practical Limitations (And Reputational Liability)
All AI content generation tools are trained using data. Lots of data. And the vast majority of that data comes from the internet.
Here’s the thing: much of the information on the internet is unreliable.
People often put things online or in books that are inaccurate or misleading — intentionally or unintentionally. Even information from reliable sources is subject to change. For example, journal articles are retracted, due to human error or intentional fraud. And new discoveries often mean that things we thought we knew were wrong.
What’s more, the available evidence suggests that AI content generators are very bad at sorting reliable information from misinformation.
For example, the data sets that the current generation of AI tools are drawing on include written and visual data that often lead generative tools to produce material that perpetuates or amplifies a wide variety of stereotypes. A 2023 Washington Post feature reported that AI image generators defaulted to a wide variety of offensive stereotypes: “attractive” people are white and young; “leaders” are men; Muslims are men who wear turbans or other head coverings; people who receive welfare services are Black; and so on. And this was after the creators of these AI tools reported having “fixed” the image generators to reduce bias.
Separate coverage, in outlets such as Bloomberg and the Associated Press, make clear that AI’s stereotyping problem is well established, and that image and text generators frequently perpetuate racist misinformation and exaggerate harmful stereotypes to extremes. Anyone using generative AI tools needs to be aware of this problem, and take care to avoid distributing text or images that perpetuate harmful stereotypes.
Image generators are also prone to making simple mistakes that can lead to reputational gaffes, which is a particular concern for anyone using these tools for marketing, public relations or other communications projects. For example, a recent online story shared AI-generated images representing student “avatars” for each college campus in North Carolina. NC State University’s entry showed students wearing Tarheel blue, standing in front of a flag featuring UNC-Chapel Hill’s logo — though the flag was rendered in Duke’s dark blue. These mistakes are not as harmful as perpetuating racist stereotypes, but would certainly ruffle feathers in North Carolina. (If you are not a college sports enthusiast, trust me, this is bad.)
The unpredictability of AI-generated text and images is exacerbated by the fact that English can be imprecise and confusing. Many words have multiple definitions (the word “run,” for example, has more than 600 different meanings). And context will only get an AI program so far. For example, the way you phrase a sentence may make clear that you are using the word “fly” as a noun. But do you mean an insect, a tent flap, a fishing lure, or a zipper?
This poses particular challenges when communicating about health, science or other technical topics.
For example, the dictionary tells us that the word “significant” means “important and deserving of attention.” However, in the context of scientific research, “significant” usually refers to statistical significance. And if you read enough journal articles, you’ll discover that some things can be both statistically significant and relatively unimportant. AI is unlikely to know which definition applies in any specific instance.
This sort of confusion also extends to proper nouns. For example, my name is Matt Shipman. But a different Matt Shipman is a voice actor for many anime shows, and a third Matt Shipman is a professional musician in Vermont. If you request information about Matt Shipman, the AI is unlikely to know which one you are talking about, or could tell you about an actor who sings about communication tools on an animated TV series.
This potential for linguistic confusion contributes to one of the biggest challenges facing AI content generation tools, which is that these tools often fabricate information.
If users ask AI content generators to write about a subject, the tools don’t always rely solely on the data that is available. Instead, the AI tools will try to extrapolate from the existing data or, from the point of view of the user, simply make things up. This is called “hallucination.”
I’ve seen AI text generators generate plausible, but entirely fictional, biographies for real people.
And text generators have become sufficiently notorious for generating fictional citations related to scientific research that Scientific Reports ran an article about the phenomenon late last year. (I know people who refer to these as hallucitations.)
Another phenomenon that comes into play here is “AI drift,” in which AI tools effectively get “dumber” over time due to factors such as changes in data distribution or user behavior. In other words, even if an AI content generator produces accurate content on a subject today, it may get it wrong the next time you ask.
For example, we know that many text generators have gotten worse over time at basic math, something computers historically have been pretty good at. While there isn’t universal agreement on what is driving this, many experts say it is likely due to AI drift.
Reliability issues have also been underscored by erratic behavior in generative AI systems that have not been traced to any single source. A good example of this was in February (2024), when ChatGPT began producing nonsensical text in response to user prompts, with users describing ChatGPT’s output as “rambling” and “insane.” The problem began on Feb. 20, and was fixed by the close of business on Feb. 21. However, it underscored that AI tools can fail or become unavailable at any time. What’s more, it remains unclear what caused the problem or when the problem may occur again.
To make matters worse for those who are tasked with communicating about health, science and other technical subjects, the problems associated with AI tools drawing on unreliable data are exacerbated when AI content generators are asked to write about new or forthcoming research. That’s because there is little or no pre-existing data for the AI to draw on.
To be clear, new research findings do not emerge from a vacuum. Science is an iterative process, and any new findings build on previous work. However, research findings are inherently new. They mean that researchers have learned or discovered something that was previously unknown. It is, quite literally, new knowledge.
For example, I write for a university. One aspect of my job requires me to work with researchers to write news releases about forthcoming research. I love this aspect of my work, because it means I am one of the first people on the planet to learn whatever it is the researchers have discovered. I can look at the journal article they’ve written, but which is not yet published, and I can ask them questions about the work to make sure that I understand it properly.
If you asked an AI content generator to write about the subject, it would have nothing to go on. The findings are not yet online. At best, the tool would notify users that it could not write about the subject. At worst, it would hallucinate, and write something that may or may not have any basis in reality.
Privacy Concerns
Last, but not least, there are issues related to privacy. Many AI content generation tools have taken at least some steps to help protect the privacy of users. However, many — if not most — of these tools still utilize user input to continue training their AI. That’s why you should not share sensitive or proprietary information with these tools. For example, there have already been instances of private company data being disclosed by ChatGPT, with Samsung being one high-profile example.
In the context of research communication, “sensitive data” may extend to things such as forthcoming research findings.
Be Cautious
All of this means users should be particularly thoughtful when utilizing AI tools. Users should be especially wary of using AI tools to generate substantive material designed to communicate about health, science, or technology.
Professional communicators — from reporters to public information officers — are tasked with communicating clearly, effectively, responsibly and accurately. And for people tasked with communicating about subjects like public health or medical care, giving people wildly wrong information can have especially serious consequences.
And it’s not solely about using AI tools to produce content for public consumption.
Many people are turning to text generators to help them understand concepts they are unfamiliar with. This is a problem. Sometimes you may get a clear and accurate explanation. But if you do it often enough, you will get an explanation that is wrong. And if you are sufficiently unfamiliar with the subject to require an explanation, you may accept the incorrect explanation at face value. (While content generators can have their uses, this is definitely not one of them!)
As for image generation tools, they face many of the same practical challenges as the text generators. In other words, there can be problems with the data they rely on — and they can also make significant mistakes.
The images AI tools draw on are:
A) only as accurate as the images they were trained on; and
B) only as accurate as the tags on the images they were trained on.
For example, if an image generator was trained using inaccurate anatomical drawings, it’s possible (if not probable) that any anatomical images it produces will be inaccurate.
By the same token, if an image generator was trained using images of Great Danes that were incorrectly tagged as German Shepherds, it’s possible that any breed-specific images it produces will be inaccurate.
In other words, image generators will often produce compositions that are subtly (or wildly) inaccurate and misleading, if not downright weird — such as producing photos of people with more fingers or legs than usual.
Does that mean these tools are useless? No.
From a writer’s standpoint, text-generating tools can be useful for getting ideas about how to open a piece or how to transition between paragraphs or concepts. Even if the text they produce is bland or clunky, sometimes seeing what you don’t want to do can help you identify what you do want to do. (Just use caution — and remember to fact check everything!)
And AI image-generating tools can be useful for creating images that are tailored to your needs. Just use caution — and remember to fact-check everything. (Also, look out for people with eight fingers!)
Also, if using AI image generators, I recommend using tools that draw solely on images that the tool’s creator has the rights to — such as image generators that were trained only on images the creator has licensed or that are out of copyright.
In the long term…
We’ll have to see how these legal cases play out. And we’ll have to see how these tools evolve over time. Can developers resolve challenges related to hallucinations? Can they solve the problem of AI drift? That remains to be seen.
In the short term…
If you are going to use these tools, do so responsibly. If nothing else, be transparent with your audience about the tools you are using.
Other than being open about your use of AI content generators, there are no well-established best practices for you to fall back on. They don’t exist yet. This is a frontier, and we are all in the process of figuring out how and whether to use these tools — even as the tools themselves continue evolving.
That means we can’t wait for someone else to tell us what the best practices are — we have to play an active role in shaping those practices.
In other words, Critical Thinking Matters.
AI is credulous. It believes everything it is told. It is not going to interrogate the data. It won’t consider what its audience knows or what the audience’s values are. It won’t know if it is explaining something in the appropriate context, or even if the explanation is accurate.
For the foreseeable future, we will need human writers, artists, editors and fact checkers to do those things.
