How to Make an AI That’s Good at Writing Stories
Relax, writers, I won’t make it and nobody takes me seriously
By MARTIN REZNY
I was told recently that the ChatGPT AI is here to steal our (meaning writers’) lunch money or something, so I tried it out for myself, and was simultaneously glad and sad that I was quite underwhelmed.
If I were to give a review of its writing as if it was a human writer, I would have to say that while it has mastered the fundamentals of writing in English language, its writing is completely devoid of personality or originality, and while its texts are structured, they don’t have any larger, overarching points. Also, when it isn’t sure, it goes for confident lying.
So, as a person, it would be a completely unimaginative undergraduate student who’s probably not going to make it, unless they get very lucky or good at cheating. A student who can read, cite, and paraphrase what the authorities sanctioned or what bestselling authors wrote, but not think for themselves, or recognize which ideas are better or worse on their own.
This kind of writer AI can only steal the jobs of the worst, most derivative or stereotypical writers, of copywriters and technical writers, or maybe of lowest-level legal assistants or programmers. More likely, it will just take over some of the most routine writing tasks that nobody enjoys doing.
Overall, as it is, this kind of AI must end up being a definite net benefit for most people, including especially writers, who will be free to spend more time doing creative writing. The thing that writers may still need to worry about is how much better a writer AI can get in the near future.
To understand what the limitations on that appear to be, one needs to have some technical knowledge of how an AI like ChatGPT works. Full disclosure, I’m not a computer scientist, but I am a media scientist, so while I wouldn’t be able to create an AI, I think I can explain the theory.
What you need to know is that AIs like ChatGPT are something called “transformers” (in this case, the robots aren’t in disguise; well except when they’re used as deepfaking decepticons). Transformers here means text transformers. They take texts and transform them into other texts.
The texts that are being transformed are those from a database of texts that the AI was trained on. Transformed means different, but similar. If all the training inputs were nonsensical, then the AI wouldn’t be able to transform them into texts that make sense, other than very rarely by chance.
There are multiple different ways in which a transformer can learn, or be trained, how to better predict which word should come next depending on which words came before, but ultimately, the AI tries something, and then it either gets “rewarded” if it’s getting better, or it doesn’t if it isn’t.
A lot of the math or formal logic involved can get pretty convoluted, and the creators of the AI may not even be able to tell what the AI’s logic is exactly for doing what it’s doing, but the ultimate goal is to develop a language model that’s predicting which words usually belong where.
To simplify it even more, either human beings tell the AI when it’s being a good boi (like when you’re training a dog), or the AI keeps running all kinds of statistics — which words are used more often, which words are more likely to follow each other, which words tend to be grouped together.
When human scientists do the latter, run statistics on texts, it’s called content analysis, and that’s something that I was actually taught at the university directly, and I believe that it is the key to understanding just how sophisticated you can get using this approach to learn how to write.
You can figure out language this way, get a language model, sure. Not a perfect one, though, because languages are based on informal logic, which tends to run into paradoxes. Rules may clash, tell you that the same string of words can mean several different things at once. But people make errors too and read identical sentences differently. AIs can get this almost right.
But a language model isn’t a writing model. With a predictive language model, you can predict that noun should be followed by a verb, because it usually is, and your human trainer confirms it. You can figure out that noun-verb pairs are the key to how most sentences make sense.
You can determine that if the verb goes first, the following sentence needs to be treated differently. You can guess with high confidence which nouns and verbs would make sense to use depending on which nouns and verbs were key in the preceding sentences. You know which words go where.
To illustrate what that means, let’s look at a couple of results that I got from ChatGPT when using mostly the same prompt while changing only a single word. The prompt I used always started with “Tell me a story about a knight called…” That was followed by three different names for the knight. The following are the first paragraphs of what ChatGPT wrote in response.
Lancelot
Once upon a time, in the medieval kingdom of Camelot, there lived a brave and honorable knight named Lancelot. He was widely regarded as one of the greatest knights in all the land, renowned for his exceptional skill with a sword and his unwavering loyalty to the king.
As you can see, Arthurian legends, or at least their synopses, had to be part of the dataset used to train the AI, because it can tell that the word Lancelot is associated with a specific place (Camelot) and a particular set of skills (one of the greatest, exceptional, loyal). Now, let’s go completely generic.
John
Once upon a time, in a far-off kingdom, there lived a brave and noble knight named John. John was known throughout the land for his unwavering courage and his steadfast commitment to justice and righteousness.
Since the combination of terms “story” and “knight” is most often associated with fairy tales, and fairy tales have common phrasing, “Once upon a time, there lived” repeats. Not surprisingly, generic John comes from a generic far-off kingdom and is generically brave, couragous, committed, just, and righteous, or a definition of average fairy tale knight.
Obama
Once upon a time, in a kingdom of peace and prosperity, there lived a knight named Obama. He was known throughout the land for his wisdom, bravery, and unwavering commitment to justice and equality.
When you use a name of an American president for the knight, suddenly the kingdom becomes one of “peace and prosperity”. Once again, the AI determines which terms are associated most commonly with the word Obama. Also, notice how the core traits of our knight now include wisdom and equality. Apparently, the AI can also distinguish between categories like location or person, as well as positive or negative connotations.
All of this is honestly pretty impressive, but it’s still only a spell checker on steroids. The AI can guess mostly accurately what categories or labels a human would put on words depending on where they are in a sentence or paragraph, it can pull facts, and it can identify genres and conventions.
What the AI doesn’t know is why particular words go where they go in any particular instance, and it can only read lines, not in between the lines. Unfortunately for the AI, it is within the ambiguities of language where real storytelling and poetry exist, where writers often play with conventions.
Real writing exists on a meta-level to language itself, it transcends it. That’s what content analysis struggles with, even when human scientists are using it to gain better understanding of discourses. If you’re guided by how often something is used or related, you can’t explain deviations from the norm.
If you’re an AI and your job is to generate text, and when all you can know is what’s commonly related, you have no way to tell when you can effectively deviate. You could try to improvise, do a random variation, but that would fail to make sense more often than it would succeed.
Human trainers could be used to judge which deviations from conventions worked and which didn’t, but that still won’t help the AI form a model of what deviations are. Subtle deviations either use word combinations that are in most cases incorrect, or they’re double meanings of normal sequences of words. In both cases independent from surrounding words.
What’s worse (for AI, great for us) is that human writers are constantly coming up with novel deviations from the norm that have never been used before and may never repeat again. Figuring those out using content analysis would be like translating a language without repeating words.
And yet, somehow, as a writer, you probably know the experience of accidentally coming up with a neologism, a new term or phrasing, that you and most other people immediately understand. There’s some fuzziness involved, sure, alternative interpretations are possible, but basically everyone who has a functional brain can tell it wasn’t just an error.
An AI can also generate a new combination of words randomly, but until humans process it and write a whole bunch of writing about it, before it exists in the context of usage, the AI cannot learn how it’s supposed to be used. It cannot tell immediately if the new thing has any legitimate uses.
Which is where my ideas on how to make a genuine writer AI come in. Like I said, real writing exists on a meta-level beyond language. I believe it may be possible to train an AI to develop a writing model, not just a language model, but the key is that the inputs must go beyond a corpus of texts.
Put simply, content analysis is insufficient. The deviations from convention cannot be explained only on the basis of information contained within a text. The important meta-information that you need to take into account are things like who the author is, when they created the text, what their life circumstances were at the time, which texts came before or after, etc.
Just like one can categorize types of words, one can for example also categorize types of authors. With access to biographical sources and news, an AI could put various labels on authors who are well-known, like whether they were young or old, rich or poor, intellectual or down-to-earth, and so on, at the time they wrote a text. Afterward, it could probably reverse that, become able to guess writer attributes from the patterns in the writing.
After all authors are categorized as well as possible, including unknown authors by comparing their writing with that of known authors, the AI could start identifying patterns of some deviations, or do a more targeted analysis for each category and combinations of categories of authors.
This would still be somewhat blunt, and still relying mostly on some form of content analysis, but it could start moving into what’s considered higher writing skills. Author analysis could be a start of giving the writer AI an author-like personality, which could be selected by choosing parameters.
For example, you could ask the AI to write a story, but as if by an author who was currently depressed, or short on cash, or recently divorced, or while they were a student, or a professor. Which brings me to the time element, or why the sequence of writing matters — some deviation patterns will be sequential, or occurring across texts in time, as a discourse.
If you asked basically any person why most stories used to have male protagonists in the past, but as time progresses, more and more protagonists are female, they would be able to figure out that it has something to do with how culture evolves. They could believe it is a good thing, or a bad thing, or something in between, but they would get it.
At this point, so much has been written on the topic by humans that even a basic AI like ChatGPT would learn about it, but what if this trend never occurred to any human before? The first human to notice would probably immediately figure it out, but in the absence of many people writing about it, a standard AI couldn’t be trained to factor that into its model.
If, however, the sequence of texts mattered in how the AI is developing its model, it should be able to detect any patterns that a historian could notice, presumably also many that humans haven’t noticed yet, and it could probably also project the trends in the evolution of conventions into the future, predicting where some aspects of style may be headed. This could lead to what would appear to be original historical or science fiction.
On a more fundamental level, a sequential analysis of something like chapters in books or acts in plays could also lead to properly categorizing story arcs, including plotline structures and character development trajectories. This is how you could make a chat bot be able to produce a script that makes any goddamn sense, as opposed to a disjointed mess.
You could then presumably ask the AI to do things like “write a historical novel about the contemporary period with a stoic character fighting against the changing times, as if it was written 100 years from now”. With author parameters, it could be “by an author in political dissent”.
If it worked right, then this AI would predict on the basis of its model that an author not liking the future would write a character resisting the changing times now as a heroic character who is in the right. Which would by itself be a category of protagonist that the AI would know how to write, even on the level of whole arcs, not just non-sequitur catchphrases.
If I understand the underlying theory and technology correctly, I believe these advances are entirely possible, just a matter of layering AI models on top of AI models, all rooted in some form of statistical analysis. It would be enhanced content analysis, at best, but it could elevate a writer AI to a level of mediocre genre writer, one who can create new characters and plots.
I do have some ideas on what something even more advanced could be, something potentially wholly transcending content analysis, but I’m not entirely sure about that just yet, and I think I’m alredy risking getting lynched by writers for betraying my own species. Although, only a few writers make a living anyway, so, how can AI steal what we don’t have?
Well, if anyone wants to chat about this, just send me a prompt.
