SyncedReview
Published in

SyncedReview

Sepp Hochreiter on Parallels Between Attention Mechanisms and Modern Hopfield Networks

Transformer and BERT language models, powered by attention mechanisms, have pushed performance on NLP tasks to ever-higher levels. Esteemed German computer scientist and inventor of long short-term memory (LSTM) Sepp Hochreiter says his attempt to explain transformers’ attention mechanisms for a lecture produced the pithy statement “a word is…

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Synced

Synced

29K Followers

AI Technology & Industry Review — syncedreview.com | Newsletter: http://bit.ly/2IYL6Y2 | Share My Research http://bit.ly/2TrUPMI | Twitter: @Synced_Global