Something beautiful that generative text models will never allow again
The film Before Sunrise (1995, dir. Richard Linklater) explores the romance between two people who meet on a train in Vienna. One, an American, must board a plane home to the United States the next morning. The other, a French student, has detoured her trip home to Paris to spend the evening conversing with this foreign stranger. The young man and the young woman fall in love as they wander the night streets of Vienna, discussing their pasts and their views on life, aware that they soon must depart, never to find each other again.
During one scene, the couple pass by a vagrant who asks them for a word. He offers to write them a poem with whatever word they give him. The young man and the young woman agree to pay him for the poem if it adds something to their life.
The word they give him is: “milk shake”.
The poem that the vagrant writes for them is reproduced below:
The film depcits a scene where a poet is given a word and generates a poem containing that word. Such poetry prompting is becoming common now in the context of generative text models such as ChatGPT, opening up poetry generation to a wide audience.
I had ChatGPT generate a poem based on a prompt that was as similar as I could make to the prompt the vagrant poet received. The ChatGPT prompt is copied closely from the Before Sunrise screenplay. To make it fair, I also set the scene for ChatGPT, since it has no eyes to see what the poet saw.
The two poems provide an opportunity to compare human-generated poetry and AI-generated poetry side-by-side. In both cases, the poem was generated based on a prompt from a stranger. Both poems were written under a requirement that it contain a given word, and both poems were written in a brief period of time.
The poem generated by ChatGPT with a similar prompt is arguably worse in every conceivable way.
ChatGPT’s poem, in its defense, has a few qualities going for it. First, the AI poem took barely less time to generate. Second, the AI poem is slightly longer. Third, the AI poem accurately employs an AABB rhyming scheme.
Does it need mentioning, though, that ChatGPT rhymed “reminder” with “reminder” in the last stanza? Or that the milkshake is inexplicably “bland”, a gustatory judgement that’s then contradicted in the succeeding line?
In terms of style, poetry, lyricality, imagery, symbolism, beauty of langauge, and ability to convey depth and nuance, the human generated poem unquestionably dominates. By literary standards, the ChatGPT output can hardly be called poetic at all, except in the fact that it rhymes. In fact, it bears more resemblance to a radio advertisement jingle.
The poet-poem in the Richard Linklater film uses the word milkshake, but it is not literally about a milkshake. It opens with imagery that isn’t drawn directly from the prompt. The poem is written from the perspective of a daydreamer who is in love. The speaker uses the imagery of delightful food to describe the feelings of rapturous beauty he feels looking into the eyes of his beloved. He concludes by describing the shared human experience of wanting to be known more fully by another person.
The poem by ChatGPT tries so hard to ensure it’s met the requirements of the prompt that it uses the word milkshake in every stanza except the opening one. The poem exceeds the requirements of the prompt by making the word that was only supposed to be included in the poem its entire locus and whole object.
The Viennese poet knows that using a word in a poem is a different task than writing a poem about that word.
Bad poetry drives out good
The merits of the Linklater-written poem clearly outshine the language-model-written poem. Nevertheless, we have to acknowledge that the existence of AI poetry generation thwarts human poets of today from securing arrangments like the one negotiated by the poet character in Before Sunrise.
It’s been noted elsewhere that AI-generated images, such as those produced by Dall-E and Midjourney, can be at times indistignuishable from images produced through the work of human artists.
By contrast, the difference in quality seen in AI-generated poetry is undeniable, in the current state-of-the-art.
It is because of this remarkable quality gap, not despite it, that impromptu poetry must bear a chilling new ambiguity.
If a vagrant asked me today to give him a word, promising a poem using that word, I would immediately suspect that he was using ChatGPT.
Suspicion of a less-than-forthright trick is hinted at even within the 1995 film from which this example is lifted. The character of Jesse leaves with poem in hand and remarks:
You know, he probably didn’t just write that… I mean, you know he wrote it, but he probably just plugs that word in. You know, whatever, “milk shake.”
Poetry can now be generated, easily, without the imprint of human emotion. The challenges for us as consumers of poetry can be summarized as follows:
- There is a difficulty in distinguishing bad poetry generated by a person from bad poetry generated by a machine. We might, given human sympathy reward even a bad poem if we feel that there’s something noble and uplifting in the attempt at crafting poetry. If we were to find out that a vagrant-poet was not exploring the expressiveness of his soul but instead exploring the expressiveness of a free online tool, would our visceral reaction to him differ?
- There is a fear of an experience where you listen to a poem and realize that you’ve just wasted your time listening to an emotionally vacuous Markov word-chain. This fear — that we will look for meaning in the words of a stranger and find nothing worthwhile — isn’t new. Most of us contend with this struggle regularly as we decide whether we are being advertised to, or, if we know we are, the ad’s level of tolerability. This can happen in other contexts as well, such as listening to the speech of a person in the throes of dipsomania. However, most of us have become familiar with the locale of advertising and of the tell-tale signs of drunken speech. AI-generated language content has not yet been well-delimited or well-demarcated.
- There is a greater dread that AI-generated content could move us to tears, wonder, joy, even ecstasy, and that our sensibilities can be so easily manipulated by a reproducible robotic process. As text generation gets increasingly sophisticated, we may observe the quality gap shrink to the point where a ChatGPT production matches or exceeds the quality of the Linklater poem. If we can be filled with spirtual truth or meaningful wisdom through a process deployed on resilient, high-availability compute clouds, controlled by profit-seeking capitalist enterprises, what hope have we of determining our own life trajectories through reason or the detection of beautiful feelings hinted at behind the words of dreamer-poets?
