Generating Harry Potter fan fiction
In this post we will explore Recurrent Neural Networks (RNNs) applied on Harry Potter books to learn and generate harry potter fan fiction.
Here’s a snippet of the final result:
Ron had hooked all out of the dormitory and the heat.
“It means of Portkeys, Ron asked, envisaging the same thing he could not be returning. As they entered the Great Hall for the first time in France, yes, he was going to be disqualified for delight at the end of the memory; he felt wide-awake — wide-awake, and worried.
Three days ago he had been damaged, a connection in the West Country that had happened.
“How was you doing here? How did you know?”
“We sent messages to reporters in the U-bend, thinking about Wonky-Faints.
It is the only one You-Know- oh, all right, I don’t think so,” said Harry, shrugging.
Looks like it could be part of one of the books, no? :)
How do RNNs work?
RNNs have the unique ability to take an input of any size and produce outputs of any size that look remarkably similar to the inputs. In the case of text, RNNs can take a list of words (ex: books) and produce a list of words that looks just like the inputs.
The type of RNN most frequently used in practice is Long Short-Term Memory (LSTM) and that is what we will use for this post.
LSTMs assume that words that appear closer to each other in the input corpus are semantically related. During training the go through batches of words to learn about these correlations to build a model of the input as a whole. When primed with a starter word the LSTMs can then predict a sequence of words based on this model. I highly recommend reading “Understanding LSTM Networks” post by Chris Olah to get further insight into how they work.
If you want to follow along you can get the code for this post here: https://github.com/malhotraa/word-rnn-tensorflow
How to train a LSTM?
- Clean and preprocess raw text (utils.py)
- Build vocabulary of words (utils.py)
- Map training data from words to numbers based on vocabulary (utils.py)
- Chunk training data into batches (utils.py)
- Create multi-layer LSTM model (model.py)
- Train LSTM model for N epochs saving model after every epoch (1 epoch = 1 pass over training data) (train.py)
- Predict words in succession based on a starter word taking into account previously predicted words (sample.py)
What comes out?
Starter word: “Dumbledore”
Dumbledore to the back of the dungeon, and looked up at the foot of the room.
“To the Prime Minister of the Regulation and Control of Magical Creatures. Play to the mercy of the Carrows. …”
Now Draco Malfoy, Crabbe, and Goyle, who were dazzled by the scruff of the nearest pillar.
A crimson bird the size of a postage stamp, which was now rummaging in the middle of the classroom.
“Ready?” said Lupin, who looked as though it had shivered.
“Fawkes?” Harry breathed, “Dumbledore!”
**********
Starter word: “Voldemort”
“You transported Peeves grasping the door and disappeared through the door to bed.
“Well?” murmured Dumbledore.
“Karkaroff’s thing he said. “Classy. You don’t want you to get rid of it?”
Dumbledore looked at him beseechingly, but she shook his head too.
“And Hermione,” said Harry, shrugging.
**********
Starter word: “Ministry of Magic”
Ministry of Magic at the end of the Order of the Gaunts’ house. It had once belonged to Harry alone.
“It’s fine,” he said quietly, as they left the infirmary and landed in the middle of the room.
“To the Prime Minister of Muggles. Urgent we meet. Kindly respond immediately. Sincerely, Fudge.”
The man in a clear, deep-blue depths of the West Ham colors. “Good holiday?”
“Not bad,” said Harry, staring at the Prime Minister.
“Er,” said the Prime Minister, “listen. … I don’t think so,” said Harry, shrugging.
Many thanks to hunkim for posting the original version of word-rnn-tensorflow code.
If you have any questions, please post them in the comments.
