Teaching a neural network to love
--
A few of my friends are getting married this summer, and I wanted to send them off with the wisdom of the brightest minds and most celebrated romantics in history. So, I trained a neural network on over 60,000 quotes on love, life, and relationships, and asked it to generate some ideas of its own. Here are my favorite machine-made quotes- after, I’ll share how I did it!
How it’s done
Training a neural network is almost absurdly easy for a user. It’s more my experience as a computational modeler to have 10,000 lines of code describing the biophysical properties of some system, so it was a tremendous shock to realize that a surprisingly powerful neural network (NN) can be implemented in only a few lines of code. An NN like the one I used can quickly learn the complex statistical structure of an input — like text, or images — using only a rote application of matrix algebra and some optimization. I’ve been playing with NNs for only a few weeks, so that should indicate how simple it can be to start having fun. Largely, that’s thanks to the amazing resources available online.
I used something called a character recurrent neural network, which is a type of NN that trains on text data- you might know it from the popular blogger Lewis and Quark, who names guinea pigs and paint colors! The software I used is a flavor of Andrej Karpathy’s code, a sort of turbo-charged version developed and hosted on github by Justin Johnson.
The most involved part for me was finding the training data and formatting it so the NN could start learning. Goodreads.com turned out to have a wonderful archive of quotes tagged by topic from lots of authors, philosophers, and other luminaries. There were over 50,000 quotes tagged “love”, and then a few other related categories (“relationships”, “marriage”). I built a webscraper in Python to crawl through all the pages of quotes, extract each one and save the whole corpus in a single text file. Next, I instructed the NN software to fit a 3-layer, 256-node model to the corpus. I elected not to use the optional GPU modules to speed up the process, since I haven’t installed the Titan graphics card sitting on my shelf yet.
Then I went to bed. When I woke up, it was 26,000 iterations later. I sampled the trained model and…voila! New text came rolling out, expounding on the human experience of love in all its tragedy and glory.
Most of the output gives you the vague feeling of having read something, but being totally unclear what it meant. Like this:
His passion in the wild could be happiness that happens to a terrority of a choice. It was the truth, the same world of the darkness… the consciousness, we call on the way to be so soft.
There are plenty of phrases that run on and on and on…
“If I will never forget the Dan around him, all of the best with an emotional passionate life between so won’t be hurt of loving you were heart and say the less success of the bathroom and present sees in the world, he realized that means when it’s a form of yourself a decades?
And a lot that defies any Chomskian intuition about grammar.
Maybe why you’re ever its home. You get hurt all the other!
Sometimes the NN got a little frisky.
This way of our lives, energy for a monogamous sex, and thought that had caught lightly.
Dan just smacked her growling of desires of her own, but start in love with it.
Seriously, who is Dan? The NN brought him up at least twice, so he must be special. Perhaps the machine really can feel love.
