Sentiment analysis using RNN
We have seen how to predict sentiment of text using fully connected neural networks (NN) in this story. Here we will use the LSTM which is a recurrent-neural-network(RNN) to analyse the sentiment of texts.
The task performed in this story is open-ended, but initially we will preprocess the data, train embedding and then build models to predict sentiment of the reviews. To follow along, this notebook can be used. This work can also be reproduced using the kaggle-cloud instsance
First of all we get rid of the extraneous punctuation, then we will tokenize the words in text so that we can embed them as numerical values.
We start by creating a map of words to numeric value, and then using those numeric values to tokenize a review. The following code snippet performs preprocessing
Similarly we tokenize the labels. Then we remove the reviews that are very small or very large(the outliers) and we truncate or pad some of the reviews as necessity arises.
We have decided our feature length to be of 200. The following code snippet achieves stated purpose.
Once we have created the features, we split the data in training, validation and test set. Following code snippet achieves stated purpose.
We will use TensorDataset, followed by Dataloader to iterate over the training, testing and validation data for purpose of model development.
We will define our model, following image shows the model-architecture
To develop our model, we will create an embedding layer, followed by an LSTM layer, then we will create a fully connect layer and finally we will use sigmoid layer to predict our desired output. Following code-snippet shows the model definition using PyTorch.
We instantiate the model using following hyperparameters
Finally training the model for 4 epochs, yields Test loss: 0.423
Test accuracy: 0.822.
This story was developed as a part of self-learning exercise for the Deep-Learning nanodegree, for which scholarship was provided by Udacity and Facebook. I thank https://twitter.com/cezannecam for explaining the underlying concepts. Contents of the above stories are borrowed from the contents provided by Udacity for the stated course.