Deep NLP: Sequential Models with RNNs

When we are using NLP to deal with textual data, one key point we must understand is that the data is always in the form of sequences and the order of the data matters. For any given sentence, if the order of words is changed, the meaning of the sentence doesn’t stay the same, hence we can say that the sentence information is stored in both the words as well as the order of the words in that particular sentence. In any type of data, if the sequential order matters, we call it sequential data.

Traditional neural networks typically cannot handle sequential data. This is because when we build a neural network for a particular task, we need to set a fixed input size at the beginning, but in sequential data, the size of the data can vary. A sentence can contain 5 words, or 20 words, hence we cannot configure a neural network to effectively deal with this kind of data. Even if we were dealing with sentences with the same number of words, which is an ideal scenario, when we input the processed words into a neural network of some fixed input size, a neural network is not designed to pay attention to the sequence of the words. The model will effectively learn from the semantic information of the individual words in the sentence, but it will fail to learn from the order of the words in the sentence.

To convert textual data into numerical format so that we can input them into neural networks, we must convert them into vectors. These can be either one hot encoded vectors or word vectors. I have explained about these in the previous…