Deep Learning: Word2Vec and Embedding

In this publication, we will continue the Introduction to Deep Learning talking about the concepts behind Word2vec and Embedding.

Word2vec

This is a technique used in neural networks to classify texts, such as positive or negative reviews of a film.

In this technique, words are transformed into feature vectors, that is, each word becomes a vector of weights that define its characteristics.

Example of a vector of weights for the example word.

To use Word2Vec you need to implement Embedding.

Embedding

Embedding increases network agility by limiting its inputs so that only the words used in a given input pass through the network.

Its implementation is to transform the words into indexes according to their vocabulary and to pass the matrix containing all the indexes of the words used in their input text by the embedding layer.

Word2vec process.

Embedding Layer

The Embedding Layer is a layer composed of an array with a number of rows equal to the number of words in the vocabulary and with a number of columns equal to the number of features of the words.

It is in this layer that the words are finally represented by the vectors of weights.

Embedding layer matrix.

From this point, the words that make up the text are already transformed into resource vectors, allowing numerous possibilities of implementations related to the interpretation or classification of texts.

I hope this article helps you understand a few more concepts of Deep Learning! Until the next!