Open-domain question answering with DeepPavlov

Vasily Konovalov
Feb 4, 2019 · 6 min read
Image for post
Image for post
Image for post
Image for post
Figure 1. A typical dialogue with an ODQA system

Model description

The architecture of the DeepPavlov ODQA skill is modular and consists of two components: a ranker and a reader. In order to answer any question, the ranker first retrieves a few relevant articles from the article collection, and then the reader scans them carefully to identify the answer. The ranker is based on DrQA [1] proposed by Facebook Research. Specifically, the DrQA approach uses unigram-bigram hashing and TF-IDF matching designed to efficiently return a subset of relevant articles based on a question. The reader is based on R-NET [2] proposed by Microsoft Research Asia and its implementation by Wenxuan Zhou. The R-NET architecture is an end-to-end neural network model that aims to answer questions based on a given article. R-NET first matches the question and the article via gated attention-based recurrent networks to obtain a question-aware article representation. Then, the self-matching attention mechanism refines the representation by matching the article against itself, which effectively encodes information from the whole article. Finally, the pointer networks locate the positions of answers in the article. The scheme below shows DeepPavlov ODQA system architecture.

Image for post
Image for post
Figure 2. The DeepPavlov-based ODQA system architecture

Ranker configuration

The ranker configuration files are located in the deeppavlov/configs/doc_retrieval folder.

Reader configuration

The reader configuration files are located in the deeppavlov/configs/squad folder. The reader component aims to find an answer to a question in the given articles retrieved by the ranker component. The reader implementation is based on the R-NET reading comprehension system developed by Microsoft Research. The detailed description of the reader configuration parameters can be found in the DeepPavlov documentation.

Model requirements

In order to interact with the model, first install the model’s requirements.

python -m deeppavlov install deeppavlov/configs/odqa/en_odqa_infer_wiki.json

Interaction with the model

You can interact with the models via the command line:

python -m deeppavlov interact deeppavlov/configs/odqa/en_odqa_infer_wiki.json -d

Training the model

Both components of the ODQA system should be trained separately. However, if you want to run ODQA on your data, you need to fit only the ranker component. The reader component is pretrained on the Stanford Question Answering Dataset (SQuAD). SQuAD is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable [3]. That said, the DeepPavlov documentation fully describes how to train the reader component on your data.

Conclusion

In this article, I’ve described the ODQA model of the DeepPavlov framework. This model is based on a two-component approach. While designing an ODQA architecture, you should always seek balance between the model’s performance and its resource requirements. By using our system, you can achieve good performance with reasonable resources. Nowadays, all the best-performing models are based on the BERT architecture (Bidirectional Encoder Representations from Transformers) [5]. However, the BERT-based systems require huge computational resources.

Image for post
Image for post

References

[1] Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017). Reading wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051.

DeepPavlov

An open-source library for deep learning end-to-end dialog…

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store