Sentiment Analysis with pretrained model using Apache MXNet C++ API
Introduction
Apache MXNet offers C++ API that is closer to MXNet implementation and do not have overhead of interpretable languages like Python. The MXNet Python API is a front-end API for the MXNet back-end which is implemented in C/C++. The MXNet C++ API is also a front-end API. However, it provides user-friendly interface to the MxNet back-end. Since the MXNet C++ API interacts with back-end natively, it does not have add significant overhead. By using these API, it is possible to implement inference applications that can be run on resource constrained environments. Due to C++ API’s tighter integration with the back-end, the inference applications can achieve higher performance in terms of throughput.
In this blog, we will see how these API can be used to develop a simple application that will load a pre-trained model and perform sentiment analysis on input lines of text.
Prerequisites
Pre-built MXNet C++ API
The MXNet C++ API is part of Apache MXNet GitHub repository. For using this API, follow the instructions to build the Apache MXNet from source along with C++ API.
Pretrained RNN Model
For the example discussed in this blog, we will use an RNN Model that is trained on the IMDB data set. The RNN model is built by following the instructions in the GluonNLP Sentiment Analysis Tutorial. This tutorial shows how we can use GluonNLP to build a sentiment analysis model whose weights are initialized based on a pretrained language model. Using pretrained language model weights is a common approach for semi-supervised learning in NLP.
Few words about GluonNLP: GluonNLP provides implementations of the state-of-the-art (SOTA) deep learning models in NLP, and build blocks for text data pipelines and models. It is designed for engineers, researchers, and students to fast prototype research ideas and products based on these models. To get familiar with Gluon, you can refer to 60-minute Gluon crash course.
Gluon can also act as bridge between various langugage bindings (such as Python, C++, Scala, Java) that are supported by MXNet. For example, as shown in this blog, we have developed, trained the model using Python API and exported the model that can be loaded using MXNet C++ API.
Coming back to the example, the tutorial uses ‘standard_lstm_lm_200’ available in Gluon Model Zoo and fine tunes it for the IMDB data set.
The model consists of
- Embedding Layer
- LSTM Layers with hidden dimension size of 200
- Average pooling layer
- Sigmoid output layer
The model was trained for 10 epochs to achieve 85% test accuracy.
The model files can be found here
Access to word dictionary
While training the model we tokenized the input training set to produce a dictionary also known as vocabulary. This dictionary contains index representation of each word that has occurred in the training data set.
We need to have access to this vocabulary for converting the input text into its index form.
The vocabulary or dictionary that was used while training the above model is here sentiment_token_to_idx.txt. Each line of the dictionary file contains a word and a unique index for that word, separated by a space, with a total of 32787 words generated from the training data set.
Developing the application to perform sentiment analysis
With all the prerequisites addressed, we can start developing the application to compute the sentiment score for the input text. In order to keep the blog post simple, we will focus on the important steps in the application. For the complete program please refer to sentiment_analysis_rnn.cpp file.
Step 1: Loading the pre-trained model
The model consists of 2 files
- sentiment_analysis-symbol.json: The file contains the symbol graph.
- sentiment_analysis-0010.params: The file contains model parameters after 10 epochs.
The model file can be loaded using Symbol::Load() method. Upon successful loading, the MXNet returns a symbol graph. Here is the code snippet that shows the loading of model file.
Next, we will load the model parameters. The MXNet C++ API does not provide a method to load the parameters from file. However, it does provide a method to create NDArray objects from the file. The following method shows how to read the parameters from file. The function creates a map of parameter name to NDArray network.
Step 2: Binding the model and creating executors
The “Symbol::Bind()” or “Symbol::SimpleBind()” APIs are used to bind the symbol graph obtained after loading the model file with the argument map obtained after loading the parameters. During the bind operation, we also need to specify the inputs for the model. These inputs are the names of input symbols and place-holder NDArray with the appropriate shape.
The Bind API returns the Executor object which is used later to run the forward pass.
Step 3: Creating shared executors
The executor expects an input NDarray of a fixed length. In other words, an executor can run a forward pass on the input line of text having fixed number of words. If we want to use the same executor for all the lines in the input, we would have to pad or trim each line to match the expected input for that executor. Although, with this approach we will get the sentiment score for each line, the accuracy of the score will be affected. This is because while trimming the input line, we may lose the significant words in the input.
In order to improve the accuracy we will take following approach:
- We create multiple executors each accepting a fixed number of words in the input such as 5, 10, 20, etc.
- For every input line we try to find the closest executor that will be able to process the entire line.
Creating multiple executors will require larger memory because each executor will try to allocate space to store the parameters. We can optimize the memory usage by creating shared executors. In this approach, we create an executor that will accept the maximum possible input length. The subsequent executors refer to this executor as a master executor and share the memory with the master executor.
Here is the code snippet that shows binding the model and creating the shared executors. We will create executors for input lengths [30, 25, 20, 15, 10, 5]. Each executor is referred by a key also known as bucket_key. The bucket_key represents the length of input (i.e. number of words in the line) that the executor can process.
Step 4: Processing the input line
We cannot feed the text data into the model as it is. The input line needs to be converted into a vector of indices representing each word in the line. For this representation, we load the vocabulary in a map where words are keys and values are integer representation of those words. Then, each input line is converted into NDArray of indices that represent words in that line.
Step 5: Generating the sentiment score for the input text
We can pass multiline text to this application and carry out following steps:
- We will split the input text into a set of lines separated by “.”.
- Each line in the input text will be represented as an NDArray of indices using the vocabulary file.
- We will find the most suitable executor to run the forward pass.
- The input NDArray will be trimmed or padded as per the input requirements for the executor.
- We will run the forward pass and Sigmoid operator to obtain the sentiment score between 0 to 1 for each line.
6. After all lines are processed, we will compute the average of sentiment scores produced for all the lines.
Conclusion:
The above example when run to compute the sentiment score for one line review is as follows:
$command> ./sentiment_analysis_rnn --input "This movie is the best."
[11:15:11] sentiment_analysis_rnn.cpp:420: Downloading ./sentiment_token_to_idx.txt with status 0
[11:15:11] sentiment_analysis_rnn.cpp:218: Loading the dictionary file.
[11:15:11] sentiment_analysis_rnn.cpp:176: Loading the model from ./sentiment_analysis-symbol.json
[11:15:11] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.3.1. Attempting to upgrade...
[11:15:11] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
[11:15:11] sentiment_analysis_rnn.cpp:189: Loading the model parameters from ./sentiment_analysis-0010.params
[11:15:11] sentiment_analysis_rnn.cpp:370: Input Line : [This movie is the best] Score : 0.964498
[11:15:11] sentiment_analysis_rnn.cpp:473: The sentiment score between 0 and 1, (1 being positive)=0.964498With multiline review, the example produces sentiment score for each line and then computes an average as follows:
$command> ./sentiment_analysis_rnn --input "This movie is the best. The actors have performed very well. But the script is very weak."
[12:02:39] sentiment_analysis_rnn.cpp:420: Downloading ./sentiment_token_to_idx.txt with status 256
[12:02:39] sentiment_analysis_rnn.cpp:218: Loading the dictionary file.
[12:02:39] sentiment_analysis_rnn.cpp:176: Loading the model from ./sentiment_analysis-symbol.json
[12:02:39] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.3.1. Attempting to upgrade...
[12:02:39] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
[12:02:39] sentiment_analysis_rnn.cpp:189: Loading the model parameters from ./sentiment_analysis-0010.params
[12:02:39] sentiment_analysis_rnn.cpp:370: Input Line : [This movie is the best] Score : 0.964498
[12:02:39] sentiment_analysis_rnn.cpp:370: Input Line : [ The actors have performed very well] Score : 0.998719
[12:02:39] sentiment_analysis_rnn.cpp:370: Input Line : [ But the script is very weak] Score : 0.0982834
[12:02:39] sentiment_analysis_rnn.cpp:473: The sentiment score between 0 and 1, (1 being positive)=0.687167In the summary, we can use MXNet C++ API to implement inference applications that are efficient and can run on resource constrained environments.
Reference:
