Chainer Tutorial: Sentiment Analysis with Recursive Neural Network

Introduction

In this tutorial, we will use the Recursive Neural Network to analyze sentiment in sentences.

Sentiment analysis is one of the major tasks of Natural Language Processing (NLP), which identifies writers’ sentiments in sentences. When expressing sentiment, we basically uses labels whether it is positive or negative. For example, in the case of the dataset used this time, emotions are expressed in 5 classes like 1(really negative), 2(negative), 3(neutral), 4(positive), 5(really positive).

cited from [1]

Sentiment analysis is implemented with Recursive Neural Network. Recursive Neural Network is a recursive neural net with a tree structure. NLP often expresses sentences in a tree structure, Recursive Neural Network is often used in NLP. First, we explain the training method of Recursive Neural Network without mini-batch processing. After that, as a advanced story, we explain the training method of mini-batched Recursive Neural Network.

In this tutorial, we will understand the following:

1. What is Recursive Neural Network?
2. Implementation of sentiment analysis by Recursive Neural Network using Chainer

• Training method of Recursive Neural Network without mini-batch
• Training method of mini-batched Recursive Neural Network

What is Recursive Neural Network

Recursive Neural Network is one of Recurrent Neural Networks that extended to a tree structure. As both networks are often written as RNN, so we need to be careful which one we are expressing. In many cases, it refers to the Recurrent Neural Network in many cases, but in natural language processing it sometimes refers to the Recursive Neural Network.
Recursive Neural Network uses a tree structure with a fixed number of branches. In the case of a binary tree, the hidden state vector of the current node is computed from the hidden state vectors of the left and right child nodes, as follows:

This operation is sequentially calculated from the leaf nodes toward the root node. Recursive Neural Network is expected to express relationships between long-distance elements compared to Recurrent Neural Network, because the depth is enough with log2(T) if the element count is T.

Implementation of sentiment analysis by Recursive Neural Network using Chainer

As shown below, we explain the implementation with Colaboratory. If we have a browser, we can immediately run the tutorial in the GPU environment. So, please try it!

Sentiment Analisys with Recursive Neural Network - Chainer Colab Notebook 0.0 documentation

Reference

• [1] Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank