BERT-QE: Contextualized Query Expansion for Document Re-ranking
NLP Research Paper Walkthrough
Abstract
Query expansion is one of the prominent pre-processing steps used in the area of Information Retrieval, where the task is to improve the Hit Rate of fetching relevant documents from the database for a given query. Query expansion helps in augmenting the query with relevant keywords for better performance. This paper uses BERT for doing contextualized query expansion by selecting relevant chunks from the underlying documents as possible augmentations and then selecting the documents based on the new retrieval scores.
Some of the earlier methods used the Pseudo Relevant Feedback (PRF) method to expand the query with information chunks extracted from top-ranked documents and then using the modified query to again re-rank the search results. Authors go a step ahead and find the relevance score for every information chunk with both query and document.
Method
The authors’ propose a three-step pipeline to re-rank the list of documents that are retrieved using other unsupervised ranking algorithms like BM25, TF-IDF, etc.
