The Lord of The RNN

I don’t know you, but I love the world of “The Lord of The Rings”. Not only the “The Lord of The Rings” story, but also “The Hobbit”, “The Silmarillion”, “Unfinished Tales” and so on. I’m fascinated with the fiction of J.R.R. Tolkien and the universe that he created. Beside that, I also love what Deep Learning can do. In this case, what a Recurrent Neural Network (RNN) can do.

Middle Earth

If you don’t know RNN, the idea behind this is very simple. Let’s think first about what a traditional neural network do. In a traditional neural network, one input generates an output, right? If you already have your network trained, every time you put some input in the network, the output will always be the same for that input. In this case, all the inputs and their respective outputs are independent of each other.

Tradicional neural network

Now, in the RNN, we can think that the network has a “memory” that is able to store information about what has been passed in the network. In this case, inputs that have been passed to the network early can influencing the output of a new input. So, with RNN we can make use of sequential information of the data. For more details on how RNN works, I suggest reading the following tutorial: Recurrent Neural Networks Tutorial.

Simple Example of a Recurrent Neural Network

Who don’t know about RNN and/or traditional neural network could be thinking right now why this kind of “memory” is so important. What RNN can do with this “memory”? Some common applications of an RNN nowadays are: Natural Language Processing, Speech Recognition, Machine Translation, Question Answering, Image Captioning, Generating Text and others things.

So, what is the relation between the fiction of J.R.R. Tolkien and RNN in this article? Well, as I have listed above, we can train an RNN to Generating Text, right? So, how about we train an RNN with the “The Silmarillion”, “The Hobbit” and “The Lord of The Rings” books? It would be awesome create some new lines of text that looks like the type of writing of Tolkien in these books. Imagine create a new story of Fëanor, Fingolfin, Beren and Luthien, Smaug, Aragorn and Arwen, Frodo and Sam, Legolas and Gimli, Bilbo and Gandalf, Merry and Pippin, Faramir and Eowyn, Ents and Entwives, a new fellowship of the ring and so forth (this infinity of possibilities makes me super anxious to read all again).

Hobbiton Movie Set — Author: Andres Iga

To do that, I trained a special kind of RNN called Long Short Term Memory (LSTM). The LSTM network is an RNN that can learning long-term dependencies, which is very good in this case. Again, for more details on how LSTM works, I suggest reading the following tutorial: Understanding LSTM Networks.

After training the LSTM network, I was able to generate new texts. I set, for each text, a total of 750 characters to be generated. The first text began with the “Fëanor and Fingolfin” characters and the network output this:

Fëanor and Fingolfin and Men were so at the top and saw the brothers that the stars of hard they could go had a ground of he shall. In the trees of the treps had some words; and he had said at his fear and sheer on their booled; and the trees at tell her. But a bow her stands say and the boat on the despair of the sea and and shining, and he had nearer they came to the store of a long shoping they would see the black beard, and shall see. The white was to the saunt of the song and shoulders of any stirred. ‘The woods was been a between words and wooded the with a great waters as a wood with take one of sat and saw a good.’ I called the sense of the March of Sauloth in the Song of the Silmarils of the Elvens of the Misty Ring of the Naun he shall not come on a s

The second text began with the “Beren and Lúthien” characters and the network output this:

Beren and Lúthien, and they deven at how seemed in. ‘I had been,” he said and some are that still huse here again, and seen hobbits of the song of Minas Tirith and there the shall a long first shield they seemed alone; but the still seeks and burrent, standed on the ships was a shadow about the stars in take the learn of the still of those that here, and she can hold on a ships of the time as that seen, but they shall not hear. He was a blanken already he soon, and that he were the black, and they hoursed to the star of a stream, still as the lands of his would heard and which think went the listens that the same and searth and the leaves of the stofter of the Might of the Elves of the North of the Ring of Gondor. But a golden hard seemers and as any wellow.

The third text began with the “Aragorn and Arwen” characters and the network output this:

Aragorn and Arwen; from. Bilbo has nucking if the stranges tone that as were brought him on the shape, they seemed and set. Then they was a stone tone to him, and he had sinted a middle of through the stars, that seemed to the stone as they had been stared and best.

‘I was netted in still a little andow that was all the world of his head, and have the less took were to here with the hands of Minas Tarin. But they was a laster of stand or to hobbits with him, and say of this hall of the Merry of the Noldor of the Men the listentar, and they had become of the still, but it were sharped a bear of the deal of this shore of the Stirgount in his, and then the lunged west which he shall be to somp them as he saw the time on the stoused sea, and so there were forde

The fourth text began with the “Smaug, Bilbo and Gandalf” characters and the network output this:

Smaug, Bilbo and Gandalf alone in hobbits than they deven a foring, both a should best, and to a stories seemed at our first and star at one back. There had too both and the time with his head and theys, but they heard to him were battle track there. But he walls to the holain.

‘I came up and the house though you set him. If they had sat to think, and how the white, they were things, and his hound had been became to a ground to an is the sons, and he had brought the stone, and all the wise and a shone of the hand of these sent of as had say the tankes and to his sank at a book of the three a steper, but in his same traid and was bore and all them out of his far the last and ferchour at, but the rise of to the high tale. I had some hare or belossed in the station.

The fifth text began with the “Ents and Entwives” characters and the network output this:

Ents and Entwives of Men through the Saruman. Then went and bethe was the thought; and he was buitted is beated, to see helped to her breached the thought of the Marth that hart the tines.

‘To that shall all the lisses. But the hand of the shirr than any hand was taken that the song of the Morner our time they had started and that set in the top of the horn of the Mountain, and as a bottom of a bragening wonder in the way with his things and strided of the bright. All there were a large shadow of the sound of the worm in her horses to the song of the Shire, and the stars that with the tares of the waters of his tried of hard the strish was business of the water of his top and as all the same and the white horse and seemed;

They were bundled the shadow th

The sixth and last text began with the “A new fellowship of the ring” characters and the network output this:

A new fellowship of the rings with a bank, shorter at all the world.

There was he sang oft, but at least the tarred were they wish as he had said they sat on it and hurry that the shadow they had a galrilings and here. All to the waters was been began the bear and tried in the dark bent which she stowed.

‘Then it was neas of stories that then then, best a some as they sound in that they deep and to the day is them. Bilbo were not a side to a lord, but that his shorls all the way or horn, all the like sang, and then the way, what were stay. The storiss though we said.

‘That say and horry tha the with was the what all him. As I should tell we there some soon at his fellows and his tale as all a long wars. A time was strong, and who were all them to time. He seemed, se

Ok ok, I know… this trained network can not replace Tolkien. Much of the text does not make much sense, but as we can see, the network learned how to form words and some sentences.

This is totally incredible!!! The power that the “memory” give to the network is great. Perhaps, in the near future, someone will build a better network that is able to create texts similar to Tolkien’s without anyone knowing that it was created by a computer (you can try this, would be awesome saw that).

If you want to check the code of this project, just go to my GitHub account.

Atles/The-Lord-of-The-RNN

Cheers :D