GloVe Embedding for Sentence
GloVe (Global Vectors for Word Representation) is developed by the Stanford NLP group and has given great results in providing meaning to the texts. It is a log-bilinear model with weighted least-squares and is based on the count of co-occurrence of words. For instance beagle and dog
Co-occurrence translates to meaningful analogies
Compared to the raw probabilities, the ratios of co-occurrence probabilities do a much better job to distinguish relevant words like ice and solid
Advantage of GloVe over word2vec and how it is different
The Word2vec model is based on the local window concept, uses Feed forward network and Softmax activation. It has Skip gram model which predicts neighboring words, given a center word, and in the CBOW model it's vice-versa. While GloVe is based on count-based probability on the entire global corpus (co-occurrence matrix). Therefore gives more practical meaning. It is fast in training and scalable for large corpus and works well with a small corpus
Python implementation of GloVe embedding for sentence
Here is the code for using pre-trained GloVe embeddings with 50-dimensions that are trained on word-word co-occurrences in a Wikipedia corpus of 6B tokens. The idea is to update the vector with each word in the sentence. Once all the words are covered, get the mean of the vector to get sentence embedding
Note:
Doc2Vec is a model that can understand and represent the meaning of entire documents. It takes in a bunch of documents and learns to create numerical vectors that capture the essence of each document. Unlike Word2Vec and GloVe, which focus on individual words