Man and Machine: AI as (co)-creator in storytelling
After more than half a century of research, artificial intelligence (AI) has finally entered its useful age. There is no question that advances in AI and automation are and will continue to transform our world. AI is already deployed for complex problems such as detecting heart diseases or predicting earthquakes. In our day to day lives, AI tells us what to read, watch or buy, and is even “co-parenting” our children [1].
Creative industries were considered as “untouchable” for a very long time due to the human nature of creativity. However, with recent breakthroughs AI is making great strides forward. Whether you are dealing with images, video, music, speech or text, deep learning has become the go-to solution for automation in almost any domain that can muster the required amounts of data. On top of that, AI is now starting to play an ever more prominent role in the creation of said content as well: Nvidia’s most recent hardware generation uses AI to enhance video game images [2], Adobe is incorporating AI in PhotoShop as a virtual assistant [3] and Google’s Magenta project is exploring how AI can allow musicians to extend their processes [4], just to name a few of the numerous examples.
While these types of AI can be employed in scenarios where the computer plays the role of the artist itself, the most promising examples are those where human and machine collaborate, with AI filling the role of a co-creator.
The onset of this creative AI revolution has not left textual content untouched. Already, computers are translating small texts from and to any language, and automatically formulating simple replies in our emails. While writing short texts is one thing, writing complete stories is a more difficult and complex challenge. Still, AI is making progress in the process of creating stories, as is being demonstrated by numerous examples.
Over at shelley.ai, an AI is collaborating with humans via Twitter to create short horror stories [5]. The team at Botnik used their predictive text keyboard to create a Harry Potter sequel [6].
And most notably, in 2016 ‘Sunspring’ was released, the first short film ever to be entirely written by an AI [7]. Does ‘Sunspring’ mark a new era where human writers will be replaced by AI? No, not yet. For AI to truly replicate or take over the human creative process of writers, there are still some hurdles to overcome.
At ScriptBook, we have been building AI that analyze and comprehend screenplays for decision support for several years now. By transferring key features from our decision support AI to a generative AI, we are able to take important steps towards developing a worthy artificial (co)- creator for generating stories.
State-of-the art: today’s script generators
The engine behind any text generating AI is what is called a language model, i.e. a statistical model that can predict and, by repeatedly selecting the predicted next word, write English text (or text in any other language for that matter). Nowadays, these language models are often implemented using deep learning techniques. They learn to read and write a certain type of text by reading a lot of said texts first. For example, the language model that wrote the script for Sunspring was created by providing it with dozens of sci-fi screenplays first. By processing these, it not only learnt the meaning of the various words, but also drew inspiration from for it’s story. Essentially, this approach is not very different from the one followed by Nvidia, Adobe or Magenta, when they are creating their generative AIs for images or music. However, generating convincing storylines is more challenging because it entails the element of consistency at various levels (e.g. dialogues, scenes, characters etc.)
Let’s take a look at this excerpt from the dialogue in ‘Sunspring’:
While each sentence is grammatically correct, it reads like you are missing some vital piece of required context, and are unable to grasp the message. In fact, H’s final statement in this excerpt expertly puts into words the reader’s feelings about this dialogue. We would even argue that the conversation is semantically almost devoid of meaning. This is not an isolated problem. Consider for example the following excerpt from one of Shelley’s horror short stories, in which the final sentence shows the same problem.
Furthermore, when looking at various examples, it quickly becomes clear that most limit themselves to relatively short stories. This is not without reason. These models have a hard time remaining consistent. Take for example Botnik’s generated Harry Potter chapter, in which the Death Eaters, which are supposed to be a group of murderous dark wizards, suddenly act like polite and gentle people:
Not only do these models seem oblivious about how different characters each have their own traits, they often seem unaware of time and place as well. For example, Sunspring’s single scene is headed with “INT. SHIP”, although no other elements from the short story reflects the fact that it is being played out on a ship.
Looking at these and other problems, the common theme seems to be that while these language models have learned how to write syntactically correct English text, they seem to have little understanding of how a story is told.
Beyond the state-of-the art: introducing “story-awareness”
At ScriptBook, we put the storyline central. We spent years building profound expertise in creating AI to analyze and comprehend screenplays for decision support. Our algorithms are capable of interpreting and understanding a story, which is precisely what the aforementioned generative models are missing. This strength allows us to carry over key elements from our decision support system to a generative AI, thereby developing an engine that is more capable of generating stories. At Scriptbook, we go beyond the state-of-the-art, by introducing a concept we call “story-awareness”. We do this by focusing on three separate aspects: awareness of characters and their traits, awareness of a script’s style and theme, and awareness of a script’s structure.
Character awareness
A good script generator should be aware that characters in a story are not only defined by their names but also by their traits and personalities. A character’s gender and looks don’t (usually) change throughout a story, and shifting talents and personalities should be gradual and sensible.
To us, it is obvious that the death eaters from Harry Potter can’t suddenly start acting like polite and considerate people. Likewise, professor Dumbledore is a wise and calm person, and shouldn’t be portrayed as a laughing stock or comic relief character.
At ScriptBook, our script-based algorithms are able to distinguish different characters from each other, extract their traits, likeability and even emotions. In broad terms, this is done by compiling all information on a character’s actions, words, behaviour and using all of this information to build up the DNA profile of a character. In our script generator’s language model, we basically reverse this process: every time a character acts or speaks, the model is made aware of that character’s profile. Doing this, the various individuals display more consistent behavior and personalities.
For example, consider the following generated excerpt, in which we see two characters, Harry and Dumbledore, having a conversation with each other. These characters’ ‘DNA profiles’ were initialized to be those of their equivalents from the famous Harry Potter movies. Their interaction here reveals that the generator clearly picked up on them being a student and a professor respectively.
Style and theme awareness
A good story doesn’t only have characters, it also has a setting, theme and genre as well. Your typical action-packed space movie doesn’t suddenly turn into a romantic drama halfway through. Many generative models solve this problem by only considering a very specific story type. For example, Shelley is solely trained on short horror stories, and thus only knows how to write these kind of stories. Similarly, the language model behind ‘Sunspring’ only considered science fiction scripts.
At ScriptBook, we make use of thousands of scripts spanning all conceivable genres and settings. To make sure each generated script has its own identity, we follow an approach similar to what we do for characters. Naturally, we already have different AI that are capable of extracting a script’s “fingerprint” containing these story-level parameters. Again, we invert this process by making the generator model aware of your preferred script fingerprint. This is not unlike requesting a preferred setting, theme and/or genre from a writer.
To illustrate this, watch what happens when we place our two characters in a script with identical parameters as Quentin Tarantino’s `Kill Bill: Vol. 1’. It is clear that the resulting script contains a lot more action and violence.
Script Structure Awareness
Finally, we take a look at how to better incorporate a screenplay’s structural elements into the language model. With structural elements, we mean typical elements like, for example, a script being divided into scenes headed by its setting. Another example is the existence of dialogue paragraphs headed by the name of the speaking character.
To us, it is obvious that these headers provide important context for the scene or dialogue respectively. For a typical language model, it is not straightforward that this short sentence (from potentially a few pages ago), provides such important context. To alleviate this problem, we explicitly make the generator attentive of any relevant context (e.g. scene headers).
When we generate a scene of our two characters in a train, the generator is aware of the location, and this is reflected in it’s output.
Moving forward
At ScriptBook, we understand stories. We created a generative engine by plugging parameters from our existing AI for script analysis, into a language model resulting in improved writing capabilities and story consistency. That being said, our model is still far from becoming the next generation screenwriter.
As previously stated, the most promising examples in AI are those where humans and machines collaborate. One could easily come up with use cases where our engine can prove to be a helpful co-creator for producing written content. For example, imagine an application in which the writer specifies a story’s outline, and uses the AI to write out the various scenes. The result might not be perfect, but should prove an invaluable source of inspiration.
While text generation has not seen the breakneck rate of progress as image generation, advancements are being made constantly. Every week, new research and ideas are being published, each time taking us another step closer to creating AI that can tell coherent and interesting stories. And while we don’t think that such machines will replace human writers anytime soon, we are convinced that it’s only a matter of time before AI will find its way into the screenwriter’s toolbox.
Finally, given the success of ‘Sunspring’, which was described by viewers as absurd but entertaining, one could argue the following: Why should AI generated stories replicate the human creative process? Why not let AI become the creator of unusual, absurd but fresh and highly entertaining content? Food for thought for our following blog post.
References
[1] Help your family create healthy digital habits (n.d.) Retrieved from https://families.google.com/familylink/
[2] Caulfield, B. (2018, October 12) NVIDIA CEO on How Deep Learning Makes Turing’s Graphics Scream [blog post] Retrieved from https://blogs.nvidia.com/blog/2018/10/12/deep-learning-turing-graphics/
[3] Miller, G. (2018, May 29) Creativity in the Age of Machines: How AI-Powered Creatives Will Enable a More Beautiful World [blog post] Retrieved from https://theblog.adobe.com/creativity-age-machines-ai-powered-creatives-will-enable-beautiful-world/
[4] Engel, J. (2018, March 13) Hands on, with NSynth Super [blog post] Retrieved from https://magenta.tensorflow.org/nsynth-super
[5] Barnett, D. (2017, October 31). Horror fiction by numbers: My not-so-shocking AI collaboration. Retrieved from https://www.theguardian.com/books/booksblog/2017/oct/31/horror-fiction-ai-collaboration-shelley-mit
[6] Shannon, L (2017, December 12). This Harry Potter AI-generated fanfiction is remarkably good. Retrieved from: https://www.theverge.com/2017/12/12/16768582/harry-potter-ai-fanfiction
[7] Newitz, A. (2016, June 6). Movie written by algorithm turns out to be hilarious and intense. Retrieved from: https://arstechnica.com/gaming/2016/06/an-ai-wrote-this-movie-and-its-strangely-moving/
[8] Benjamin (2016). Sunspring Final. Retrieved from https://www.docdroid.net/lCZ2fPA/sunspring-final.pdf
[9] Shelley (2017). Look Away. Retrieved from http://stories.shelley.ai/lookaway/
[10] Botnik Studios (n.d.), Harry Potter [blog post]. Retrieved from https://botnik.org/content/harry-potter.html
