Introduction to Transformer Networks — How Google Translate works | Attention Is All You Need

Have you ever asked yourself: how does google translate work? The answer might surprise you, but it doesn’t use any CNNs! They are done using transformers! And I’m not talking about the ones in the movies. Let’s dive into this translation mystery!

Introduction

Recurrent Neural Networks, also known as RNNs, are widely used in natural language processing but they are slow and can’t deal with a long sequence very well. Since they work only one word at a time it leads to problems like vanishing or exploding gradients. If you are not familiar with RNNs and NLP, I explained them in two videos. I suggest that you take a look before continuing this article!