How to do a Literature Review? — Research 101

Photo by João Silas on Unsplash

I was having this discussion with some of my friends as to how we should approach a completely new problem. And where to start. In this era of Deep Learning, where things are changing rapidly and the research gets outdated really soon, how do we keep up with the recent research and filter out the papers that are not relevant? In no particular order:

1. Recent survey paper -> Gets you a nice summary as to how people tried to build up on top of the baselines and tried to solve the problem — (hopefully) a categorization of what techniques worked.
2. Proceedings -> If you can get hold of a conference/workshop proceedings specific to your topic, you are already lucky enough. Eg. ViGIL workshop for people working on vision and language.
3. Specific researchers -> If you know a person who works on your topic of interest/tasks, try to briefly go through their Google Scholar profile. You would definitely find some datasets/papers/research that suits you.
4. The dataset paper that started it all -> get all your baselines and dataset biases. Go over the references mentioned in this paper and create a mental tick to know what was the previous literature in this regard. You also get an idea of related fields where you can borrow some ideas from ;)
5. Social circle -> your friends, collaborators and advisors — senior people. Also, never underestimate the power of social media. Most of the researchers are getting pretty active on Twitter/FB. Follow them and you will be updated about the stuff happening around the world. I have categorized all my social networks: Twitter only for research purposes!
6. Mailing lists ->Eg. corpora mailing list for NLP. Some of the researchers are active in these mailing lists and provide relevant feedback.
7. Thesis -> Definitely, it is a nice idea to refer to a thesis closely related to your project/research ideas. Maybe old-school and sometimes boring, but most effective IMO.
8. Github — “awesome-X-dataset” list: Some of the researchers (mostly Ph.D. Students) are building up dataset lists which can prove to be a very good starting point. Example: awesome-public-datasets repo.

Bonus tips:
1. Organize your research papers using cross-platform softwares like Mendeley. You can even share your annotations with your collaborators.
2. Some platforms like PapersWithCode also provide codes and a nice summary of SOTA categorized by tasks.

How to read papers? — Three-pass approach

Follow this nice thread and Prof. S. Keshav’s tips on how to read a paper.

I, personally, divide the first pass into the “0th pass” where I first read the Abstract and Conclusions plus having a bird-eye view over the figures and tables (with their captions). Only then I start reading the Introduction and follow this kind of three-passes. It gives me much more sense about the paper in general and what to expect. IMO, you can adapt this approach according to what suits you best! One of my friend’s argument:

“One thing I do is to rephrase the paper as per my understanding by second pass. During the third pass also make a mental remark of what you could do better/easier and what step would cause what issue and if that is reflected in the experiments. Developing these habits is the key. Once there, you would look at verbal communications, discussions, and many other trivial stuff scientifically/innovatively (based on your personality and liking). In other words critical thinking + trusting your own understanding (because you only have that to carry forward)”

Acknowledgments: Thanks to Shreyas, Nitika and Suriya for their discussions and suggestions!