Learning to Ask: Neural Question Generation for Reading Comprehension
Authors: Xinya Du, Junru Shao, Claire Cardie
In this paper, the authors consider the problem of automatic question generation for sentences from text passages in reading comprehension. Several key applications of question generation are in the area of education, chatbot, clinical tool for evaluating or improving mental health, and development of annotated data sets for natural language processing research in reading comprehension and question answering. In this work, to generate a natural sounding question, in contrast to all previous work, they propose to frame the task of question generation as a sequence-to-sequence learning problem that directly maps a sentence from a text passage to a question.
Model
The designed model in this work is partially inspired by the way in which a human would ask a natural question by paying attention to certain parts of the input sentence, as well as associating context information from the paragraph. As a result, they consider an attention mechanism on top of their encoder to guide the decoder in generating more natural questions. An overview of the model is depicted in the following figure (This figure is from here):
Encoder
As the encoder, they consider a bidirectional LSTM with an attention mechanism on top to extract the importance of each token in the input sentence when generating the output question in each time step. As the initialization for the decoder, they concatenate the output of the LSTMs in each direction. Furthermore, they consider an extra scenario, which provides the decoder with a context vector representing the paragraph information.
Decoder
To generate a token in each time step, they feed the hidden layer from the previous time step and the vector representing the last generated token to a LSTM layer. Then, concatenating the output with the attention signal for the current time step, they generate the next word by passing the output vector through a neural layer and applying a softmax on the top. To generate more accurate questions, they consider a beam search with length a 3 instead of greedy decoding.
Experiments
To evaluate their method, they consider the SQuAD dataset and locate the sentence containing the answer (for each question in the data) and use it as the input sentence. The result of comparing the generated questions with the actual one is presented in the following table:
Looking at the performance of the models, it’s clear that adding the pre-trained embeddings generally helps. While encoding the paragraph causes the performance to drop a little, this makes sense because, apart from useful information, the paragraph also contains much noise. Furthermore, they conduct a human study on the performance of their models outperforming other baselines. The results are depicted in the following table:
Although this work provides a very novel end-to-end approach for the question generation, and introduce many interesting techniques to address different challenges in the task, there are few shortcomings regarding the provided model. Firstly, the motivation behind the question generation task needed to be established in a more detailed way. Secondly, although their neural model outperforms the previous rule-based methods, dependency on a QA dataset for training can be considered as a drawback for their model.
