Using Machine Text Generation Techniques to Write Cool Stories
Can a machine write stories?
Have you ever looked at clouds to find shapes in them? I bet you did. I also bet you don’t actually believe that the clouds intentionally take shapes of other objects. It’s our mind that tries to find a pattern where there’s none. But its immensely enjoyable to look at clouds. Same goes for procedurally generated text.
My goal here is to show you how to generate random text based on a template. The effects of such technique can range from hilarious to unexpectedly clever, as we are not going to teach the computer how to understand human language. Instead, we will make it choose random elements to fill in parts of a template.
Hilariousness of procedural text generation: an example
Here’s an example generated text from one of my projects. You may notice it’s based on responses found on tech support forums:
Hello, Genji Shimada!
I hope you are a little kind today!
I tried fixing it using sarcasm, but didn’t get it to work because of an overflowing buffer in the Matrix. The error message is “error: no WordPress in sinister custom post type”. I will do it tomorrow. Also, your website is giving me a “653 error: chicken not working or no baboon ready to haggle”.
Have a nice day!
I find it funny ;)
Oh, just so you know. We will be using just a tiny bit of computer programming, but no previous knowledge of programming languages is really necessary. I’ll try to teach you everything that you need to start. After reading this article, you will be ready to start creating your own templates!
What is procedural generation?
The most basic definition of this technique (also called in short procgen) is:
Procedural generation is a method of creating data algorithmically as opposed to manually.
This usually involves using randomisation — as if the computer was rolling dice to choose which road to take next.
Computers are great at generating data, but not so great at understanding what they are creating. It requires a lot of work and resources to make a computer understand even the most basic sentences.
Hint: Natural Language Processing is the field which with teaching computers to “understand” human language.
The power of procedural generation
The power of procedural generation stems from the fact that we can generate unique content according to given criteria and we can never really know what will be the exact output.
This uniqueness can be compared to how numbers are chosen in a lottery. The pool of possible numbers is very small, but the number of possible combinations makes it very difficult to predict which exact numbers will be drawn next time.
This makes it possible for procgen to be applied as a creative tool in writing.
Drawbacks and limitations of procedural generation
Procedural generation isn’t perfect by any means. It can’t substitute real human creativity. It may seem a bit stiff. It can create incoherent (but funny) babble. But if we are careful and deliberate, it can be used to do some pretty cool stuff.
It’s because of how human brain works. We look for patterns and meanings everywhere, even where there are none. We try to make sense of the most sensless situations.
Even the most crazy combination of words, when put in a sentence, will cause our brains to look for meaning. This is a known Gestalt principle. Our brain dislikes chaos and tries to find meaning and structure in everything.
Hint: You can compare understanding procedurally generated content to looking for shapes in clouds. Can looking for shapes in clouds be still satisfying, even though we know there is no real “object” in their shapes?
What else can be generated procedurally?
Besides being used for generating interesting stories, procedural generation is used extensively in graphics and computer games, but can be applied to practically any type of content.
People have been using procgen for years to generate images, movies and demoscene intros. Being good at procedural generation techniques is often used as a measurement and means of showcasing one’s programming skills.
Fractals are geometric patterns which can be generated procedurally. They look strangely alien and at the same time — familiar.
Take a closer look at a flower next time and notice the patterns in the centre or how the petals are arranged. Fractals share a similarity with flowers.
In general, the world around us has some cool features that look as if they were created by a mathematician!
Entire fictional worlds can be created in computer games. Take for example any open world sandbox game. Minecraft is an excellent example of using this technique to provide players with endless possibilities for exploration and limitless amounts of fun.
Every game of Minecraft starts in an unique world with randomly generated geographic features:
There’s also another game I find extremely hilarious, Caves of Qud. It’s a roguelike game where you explore a procedurally generated world.
It uses a different technique (Markov chains) to generate text than what I’ll be showing you in this article. The game generates an entire history of a ficional world for you to explore, including some pretty hilarious books in which you can discover lore about the world you are travelling in.
Here’s a talk by one of the authors of Caves of Qud with examples of some very weird procedurally generated text:
Procedural generation and writing
We can use replacement grammars to substitute parts of a template with randomly selected pieces of text to create an unique story every time we run the generator.
We can create entire stories, including books using procedural generation techniques. We can generate deep, detailed heroes, complete with a description of what makes them work. And we can use this technique to come up with plot ideas. Procedural generation can work well for coming up with:
- names of people,
- descriptions of anything,
- song lyrics,
- business ideas,
- user personas (a tool used in UX design),
- parodies and travesties,
- RPG games,
- and anything your creativity needs!
There’s also an area of procgen applied to social media. Some people use this technique to create Twitter bots. Here are some examples.
Practical guide: Using tracery.io to generate text
As the author puts it,
Tracery is a super-simple tool and language to generate text, by GalaxyKate. It’s been used by middle school students, humanities professors, indie game developers, professional bot makers, and lots of regular people, too.
You can watch Kate Compton’s introduction to Tracery.io here:
We can use tracery to generate text, because as writers we learn to apply patterns and to our writing. Those patterns will become the basis for our story generation process.
Tracery and replacement grammars
Tracery uses grammars to generate text. A replacement grammar takes a starting symbol, and replaces it with one of several rules.
In Tracery, we use special markers to tell the computer where to put a randomly selected piece of text from a list which we have to provide. Those markers look like this: #marker#.
If you need help with learning JSON, I’m here, so please ask in the comments or on twitter.com/paulpela if you find it hard to grasp.
Tip: Here’s a great JSON tutorial to get you started.
Every story in tracery starts with an origin. Let’s use the Pixar’s story prompt as our starting point and practice filling it out and expanding it.
Hint: You can learn more about Tracery and test your own templates here.
1. Our first substitution
Once upon a time there was ___. Every day, ___. One day ___. Because of that, ___. Because of that, ___. Until finally ___.
Let’s make a list of symbols we can substitute for the first empty space in the above template, in JSON format:
As you can see, we’ve marked the place where we want the computer to put a heroe’s name from our list and place it in the template.
Tracery will go through the template and fill it in. Here’s how that worked in my case:
Once upon a time there was robot. Every day, ___. One day ___. Because of that, ___. Because of that, ___. Until finally ___.
Well… It doesn’t look too good, does it? Let’s fix the missing article by adding a modifier to our #hero# token.
2. Adding an appropriate article automatically
Tracery can add the missing article for us. It’s smart enough to use the correct form, too! We only have to change one thing in our template.
Once upon a time there was #hero.a#. Every day, ___. One day ___. Because of that, ___. Because of that, ___. Until finally ___.
Adding the .a to our marker tells Tracery to add an appropriate article:
Once upon a time there was a robot. Every day, ___. One day ___. Because of that, ___. Because of that, ___. Until finally ___.
Hint: You can let your creativity run wild now! Come up with more heroes you can add to this list. The bigger the list, the bigger the chance for getting a different story each time!
3. Branching: What did our #hero# do every day?
Now we can add a list of things the hero might have done. It works the same way as selecting our hero.
The power of Tracery stems from the fact that we can not only create flat lists of words to be placed in our template, but we can also nest templates inside other templates!
Running Tracery at this point may generate something like this:
Once upon a time there was a robot. Every day, she used to practice jiu-jitsu in a nearby village. One day ___. Because of that, ___. Because of that, ___. Until finally ___.
3. More modifiers
Since we are creating text, there’s always the question of making it grammatically correct.
Tracery comes with a list of built-in modifiers to help us with that. We’ve already seen .a in action. Here are some of the other modifiers:
- .ed: for creating past tense verbs
- .ing: for creating present participle or gerunds
- .s: for getting a plural form of a noun
- .capitalize: changes the first letter to uppercase
You’ll be surprised how well they work!
You should be able by now to come up with an idea of how to finish creating a story from this template.
I urge you to try it yourself and come up with ideas on how to fill in the rest of the gaps the same way as above.
If you want to practice, visit Kate’s online tutorial page. You will find out more about the rules used in Tracery, as there’s still a bit more flexibility and cool features for you to discover (such as saving the selected templates to use them further in the text — very useful to put our heroe’s name somewhere else!).
My own practical applications of procedurally generated text
Well, the possibilities here are limitless, actually! I’m sure you will be easily able to come up with something of your own. If you do this, please share your work with me! I’m really interested what you managed to achieve :)
I use Tracery to invent fictional characters for some of my Medium stories. I’m still quite new at this, so they are not very deep or detailed at the moment. But I’m slowly getting there. Here’s a story I used a short template to generate descriptions of fictional UX designers:
Measuring your consistency can help you in job search and making improvements to your workflow. Here are my…uxdesign.cc
Scroll to the section titled “What is the value behind benchmarking your workflow” to see this. Look for:
Kate, a UX designer from England looking for her first job.
I also use it in some hilarious experimental generators, such as the support response generator you’ve seen at work at the beginning of this article.
That’s it! You can now start to generate your own procedural texts!
If you need any help, please ask in the comments below or on Twitter @paulpela. I’m more than happy to help you get started!