Image for post
Image for post
John Christian Fjellestad –Distant road

📚The Current Best of Universal Word Embeddings and Sentence Embeddings

Thomas Wolf
May 14, 2018 · 9 min read
Image for post
Image for post
Recent trend in Universal Word/Sentence Embeddings. In this post, we describe the models indicated in black. Reference papers for all indicated models are listed at the end of the post.

Recent Developments in Word Embeddings

Image for post
Image for post
Elmo knows quite a lot about words context

The Rise of Universal Sentence Embeddings

Image for post
Image for post

Image for post
Image for post
A plot of HuggingFace’s dialogs Bag-of-Words. Bag-of-Words approaches loose words ordering but keep a surprising amount of semantic and syntactic content. Interesting insights in Conneau et al. ACL 2018.

Image for post
Image for post
Quick-thoughts classification task. The classifier has to chose the following sentence from a set of sentence embeddings. Source: “An efficient framework for learning sentence representations” by Logeswaran et al.

Image for post
Image for post
A supervised sentence embeddings model (InferSent) to learn from a NLI dataset. Source: “Supervised Learning of Universal Sentence Representations from Natural Language Inference Data” by A. Conneau et al.

Which supervised training task would learn sentence embeddings that better generalize on downstream tasks?



HuggingFace

Stories @ Hugging Face

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store