Github-style issue for Academic Papers — Towards Reproducible AI Research
Tomorrow’s research is always built upon that of today. Despite this, the issue of reproducibility still remains a thorn in the side of many researchers. How many times has it been that you wanted to dive deeper into a specific domain but the papers that led up to it are not verifiably reproducible?
“More than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments.” — Nature(not nurture)’s survey of 1,576 researchers on reproducibility in research
Nurture.ai is a collaborative platform for getting more out of AI research, and the reproducibility of AI papers is one of the core problems that we tackle.
Specifically, papers that are intentionally/unintentionally missing or vague in some details crucial to reproducibility. We have discussed with many researchers about how hard it is to get a clarification via email from the author over such issues, and experienced it ourselves as well.
Github-style issues for research papers
We want to change that through the Github-style issues for papers we recently launched. The benefits of this new feature are:
- A channel for paper readers and the authors alike to discuss important issues regarding papers
- Issues can be opened up to flag 🏁 problems with the paper, to ask questions, or suggest improvements. Authors will be notified about all issues opened up on their papers
- A community that becomes a natural quality control layer before these issues are brought to the authors’ attention, through the use of likes and engagement of an issue
Some examples of issues include:
- ambiguous / not clearly explained concepts in the paper
- problems in replicating results of the paper
- errors (in mathematical equations , definition of technical terms etc)
- invalid assumptions
- related works that are not cited
Opening Issues of your own
Opening an issue on a paper is simple. Simply navigate to the desired paper page on Nurture.ai, click on issues and then click the Create a New Issue button. You’ll get to fill up the title, description as well as choose a type of issue you want to open.
After that, the Nurture.ai team will communicate these issues to the authors of the paper. While anyone can open up an issue on a paper, only verified authors of papers are able to close a specific issue on a paper.
Got some feedback or suggestions?
We would love to hear your thoughts on the current platform and what we are building, as well as any problems you face in learning AI or conducting AI research — our team is committed to making this the best possible AI research platform for you.
Comment below, click on the “Feedback for Nurture.AI” icon on the bottom-right corner of the platform or email me at jameslee@nurture.ai. We look forward to hearing from you, and don’t forget to clap if you like this!
Find this useful? Feel free to check out our other initiatives AI Saturdays and #APaperADay. 😄
Nurture.ai is a collaborative platform to annotate, discuss and submit code implementations for research papers on Artificial Intelligence.
