Advances in NLP in 2017

Trends

Attention Is All You Need

Transformer Architecture
Attention used in the paper.
Attention mechanism from Neural Machine Translation by Jointly Learning to Align and Translate

Parallel Decoder for Neural Machine Translation

Non-Autoregressive Decoding
Picture from Neural Machine Translation by Jointly Learning to Align and Translate

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store