Augmenting Creative Storytelling with Generative AI

Artificial intelligence can be used to automate several aspects of the pre-production and planning stages of content creation.

It has taken away the tedious aspects like script breakdowns and budget estimations, but what’s been missing is how to sharpen the narrative and elevate the creative process.

All the above starts with a script. But how about AI augmenting the writing and storytelling process itself? The short answer is yes, AI can augment the creative process. But creating a complete work that consistently has meaning without human intervention is much further away than we think.

Over the last year, I’ve set out to meet with hundreds about this pain point — everyone from film production companies to the Fortune 500 and I keep hearing the same issue: I didn’t get into content creation to spend so much time on pre-production and not enough on the creative elements of storytelling.

That’s why today I’m thrilled to unveil RivetAI.

I’ve assembled a crew of data scientists and engineers who cut their teeth in Hollywood and worked at SpaceX, Caltech and Microsoft Research and more to launch the very first suite of AI tools for storytellers.

Kyle Wiggers from VentureBeat, who I gave a sneak peek to before launch says: “It helps people to create content much faster,” Ray told VentureBeat in a video conference, “by using AI to augment creativity. Everything starts with data.”

It’s through RivetAI that screenwriters can now pitch their work more effectively; line producers can take on more roles; production studios can streamline the RFP-creation process; and corporations can create compelling in-house and social content.

AI-assisted creative story writing needs 3 primary modules and here’s what we’re setting to accomplish:

1. Natural language understanding, leading to generation capabilities.

There are new examples in the news everyday of AI systems being trained on a corpus of text, and then creating original material. One approach that worked quite well was training a particular deep neural network known as a Long-short term memory model, that was originally invented by Jurgen Schmidhuber in the 90s. This model preserves longer-range consistency and could produce interesting sentences.

Closer to home, for SUNSPRING (produced by End Cue), this type of model was trained on a large corpus of sci-fi and horror movie scripts using unsupervised learning. As opposed to supervised learning, no goal or objective function is defined for the neural network. Then the trained deep neural network was sampled to produce a script with the seed word “sunspring”. Imagine a more sophisticated version of running a predictive text, like you have on your cell phone now, and selecting every suggestion that came up.

Natural language generation now has matured to the point where meaningful sentences can be generated that have proper grammatical structure, that helps people become better writers.

2. Understanding context, human interactions and narrative structures.

But a story is a lot more than a bunch of proper sentences that are all consistent with each other. Context is extremely important. Phrases can have completely different meanings based on the context, both the general theme or genre of the narrative, as well as the story told thus far.

Some of the very recent developments in deep learning, that we have adopted are hierarchical models and autoencoders. Hierarchical models are what the name suggests: we can construct a hierarchy of meanings, which is particularly well suited to natural language generation. With hierarchical models, like hierarchical long-short term memory networks, we can ensure that each word generated is consistent to the sentence, within a paragraph, within a scene, to within a story.

We experimented with these two developments in creating our own take on Mystery Science Theater. We trained our hierarchical long-short term memory autoencoder neural network on obscure movies and their comments and reviews on Reddit. And for a scene in a movie, like Deathstalker 2, we can extract the context from the scene description and what the characters say. That goes in as the input. And in response, we generate a phrase or sentences. We have a short clip from that movie below that our test audience found hilarious, and Mike Brown from Inverse called “Not bad”.

To go from witty dialogues and quips, to a script that follows a storyline, we needed to extend the hierarchical autoencoders to take in the context from a knowledge graph. The figures below shows an example of how we can construct a character interaction and sentiment graph from a script.

Each node in the story graph depicts the characters and the sentiment of their interactions. The graph overall depicts scene-by-scene the evolving characters and their interactions throughout the story.

Human writers, directors and producers can now specify the high level storyline through a knowledge graph. We can thus encode knowledge about the changing dynamics of various characters, their personalities, and how that evolves through interactions with other characters or life events.

3. An embodiment to grasp the human experience.

We analyzed a large number of storylines and narrative arcs in the form of scripts and articles in the public and private domains. And we generated knowledge graphs from them. Interestingly, they show some repeatable patterns, especially among the successful stories. It seems that human beings resonate with some known archetypes, and our fundamental nature has not changed since the title Aristotle wrote the Poetics. Although the various embodiments and the gestalt has.

What does this mean for a completely AI generated story? While we can score storylines against known archetypes, and in the future even suggest narrative changes to improve reader engagement, human element is vital in the creative process. Our years of living in the physical and social world helps us grasp meaning and understanding that is vital to storytelling. And that visceral and emotional knowledge that we acquired is hard to encode in an AI.

It has been an exciting journey working with my team, and our wonderful users, to provide a platform that makes video production easier, starting from even the figment of a script. And in our vision, we are continuing to work on our AI to equip content creators with superhuman creativity. These are exciting times ahead, so stay tuned for what’s to come.