Intro to Doc2Vec | Towards AI

An Intuitive Introduction to Document Vector(Doc2Vec)

Manish Nayak
Jun 24, 2019 · 3 min read

Introduction

Doc2Vec is an extension of Word2vec that encodes entire documents as opposed to individual words. You can read about Word2Vec in my previous post. Doc2Vec vectors represent the theme or overall meaning of a document. In this case, a document is a sentence, a paragraph, an article, or an essay, etc.

In Doc2Vec, the name of the document, like file name or file ID will be the input, and the sliding window of the words from the document is the output.

Similar to Word2vec, there are two primary training methods, Distributed Memory Model Of Paragraph Vectors (PV-DM) and Paragraph Vector With A Distributed Bag Of Words (PVDBOW).

They are variations of the CBOW Models and Skip Gram Models, which use to train Word2Vec, it as extending the idea of context to paragraphs by adding a label or a document-ID.

  • The Distributed-Memory Model closely resembles the CBOW model of Word2vec. This model tries to predict a target word given its surrounding context words with the addition of a paragraph ID.
Distributed-Memory Model
  • The Distributed Bag-Of-Words Model based on the Word2vec skip-gram model, with one exception instead of using the target word as the input, It takes the document ID as the input and tries to predict randomly sampled words from the document.
Distributed Bag-Of-Words Model

The input filename is not much important. Output words are essential and need to keep track of them as they came from the same document. All these words are connected to that filename.

Doc2Vec model can predict the document’s words based on its filename, the Doc2Vec model knows which words go together in a document.

The content of documents usually about one thing, for example, many different positive words are used in positive reviews, and many negative words are used in negative reviews.

After training, we can have a new document for which we need to find its document vector. Doc2Vec uses the word similarities learned during training to construct a vector that will predict the words in the new document.

Once we get that new document vector, we can compare this document vector with other document vectors and find which document vectors from the past are the most similar.

So, Doc2Vec can use to find similar documents. This will help us to find the positive and negative reviews because, in general, document with the positive reviews will have related vector and document with the negative reviews will have related vector.

Accompanied jupyter notebook for this post can be found on Github.

Conclusion

We know the importance of vector representation of documents, for example, documents clustering or classification tasks, the documents need to be represented as a vector.

I hope this article helped you to get an understanding of Doc2Vec. It will also provide inside of Doc2Vec representation of documents, and its uses.

Manish Nayak

Written by

Machine Learning, AI & Deep Learning Enthusiasts

Towards AI — Best Artificial Intelligence Publication

Towards AI, is the world’s fastest-growing AI community for learning, programming, building and implementing AI.

More From Medium

More from Towards AI — Best Artificial Intelligence Publication

More from Towards AI — Best Artificial Intelligence Publication

Data Scientist Coding Exercise

More from Towards AI — Best Artificial Intelligence Publication

More from Towards AI — Best Artificial Intelligence Publication

3 Ways Linear Models Can Lead to Erroneous Conclusions

More from Towards AI — Best Artificial Intelligence Publication

More from Towards AI — Best Artificial Intelligence Publication

Image Filtering

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade