Let machine read and summarize research papers for you
A deep learning technique can generate comprehensible summary from a research paper.
With a lot of specific technical terms, a reader who does not have the scientific background might find a research paper challenging to understand. A team of scientist from MIT Physics Department developed a deep learning technique which might help writers and researchers understand a scientific paper with a readable synthetic summary.
Rumen Dangovski and Li Jing described their work in the journal Transactions of the Association for Computational Linguistics, one of the leading journals in Natural Language Processing (NLP) area.
“We observed that reading the first 1,000 words from the research paper is generally enough to generate a meaningful Science Daily-style highlight”, written by Dangovski et al. (2019). They prepared over 50,000 full texts of scientific research papers, paired with a corresponding news-style article from Science Daily. After training their deep learning model with this large corpus, it can generate a plain language summary of a scientific paper given.
Build better voice apps. Get more articles & interviews from voice technology experts at voicetechpodcast.com
For synthetic tasks, existed models cannot efficiently recall relatively long input. For instance, Long Short-Term Memory (LSTM), one popular variation of Recurrent Neural Networks (RNNs), have excellent performance on capturing potential repetitive patterns in natural language input. However, specific keywords and technical terms would repeat many times in Abstract, Introduction, and Conclusion of a research paper. So, when it comes to summery generation task for research papers, LSTM tends to render duplicate text many times. Therefore, interpreting a research paper with existed models is quite tricky.
To cover the shortage, the researchers proposed the Rotational Unit of Memory (RUM), a novel representation unit for RNNs framework. RUM leverages the idea of rotations, which have fundamental applications in physics, to create the associative memory. It turns out that the RNNs model with RUM unit renders much more readable summary from an input research paper. Instead of repeating technical terms and keywords, the summery it generates is in a logical sequence of telling a story. For example, the summery would first describe the critical finding of the research paper. Then it goes to the experiment results that support this finding. Furthermore, RUM also yields state-of-the-art performance on various synthetic tasks.
“We have further demonstrated that our model outperforms conventional RNNs on synthetic and on some real-world NLP tasks.”, wrote at the end of their paper, “In future work, we plan to expand the representational power of our model.”. With the capability of long-term memory, RUM can work as a better fundalmental unit in RNNs than traditional Long Short-Term Memory framework. Therefore, it also has potential in other deep learning methods such as deep reinforcement learning and transfer learning.
Reference journal article:
Dangovski, R., Jing, L., Nakov, P., Tatalović, M. & Soljačić, M. 2019, ‘Rotational Unit of Memory: A Novel Representation Unit for RNNs with Scalable Applications’, Transactions of the Association for Computational Linguistics, vol. 7, pp. 121–38.
