Machine Learning is Fun Part 5: Language Translation with Deep Learning and the Magic of Sequences

Adam Geitgey
Aug 21, 2016 · 16 min read

Making Computers Translate

We just replace each Spanish word with the matching English word.

Making Computers Translate Better Using Statistics

Training data is usually exciting! But this is just millions and millions of lines of dry government documents…

Thinking in Probabilities

Step 1: Break original sentence into chunks

Step 2: Find all possible translations for each chunk

Even the most common phrases have lots of possible translations.

Step 3: Generate all possible sentences and find the most likely one

Statistical Machine Translation was a Huge Milestone

The Limitations of Statistical Machine Translation

Making Computers Translate Better — Without all those Expensive People

Recurrent Neural Networks

Humans hate him: 1 weird trick that makes machines smarter!
This is one way you could implement “autocorrect” for a smart phone’s keyboard app

Encodings

I love this dumb gif from CSI so much that I’ll use it again — because it is somehow manages to demonstrate this idea clearly while also being total nonsense.
These facial feature measurements are generated by a neural net that was trained to make sure different people’s faces resulted in different numbers.
This list of numbers represents the English sentence “Machine Learning is Fun!”. A different sentence would be represented by a different set of numbers.
Because the RNN has a “memory” of each word that passed through it, the final encoding it calculates represents all the words in the sentence.

Let’s Translate!

Building your own Sequence-to-Sequence Translation System

The Ridiculous Power of Sequence-to-Sequence Models

Image from this paper by Andrej Karpathy
Example from http://cs.stanford.edu/people/karpathy/deepimagesent/rankingdemo/

Adam Geitgey

Written by

Interested in computers and machine learning. Likes to write about it.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade