Guidelines to Write Useful Academic Reviews

Recently, I had the opportunity provide inputs to improve the academic reviewing process in a CS area. I used this opportunity to jot down what should reviewers do to provide useful reviews.

  1. The review should be limited to the merits and limitations of what has been done in the effort and described in the paper. It should not consider what could have been done.
  2. For any alternatives (e.g., methodology, techniques, data sets, visualization) that a reviewer thinks should have been used, the reviewer should provide a sound argument why the considered/described work is incorrect and why the alternative would be correct. Such feedback should provide references to the alternatives.
  3. When approaches with varying degree of accuracy exist (e.g., in case of evaluations), the review should not penalize the paper for not using the next best approach (to what has been used in the paper) if no prior work has used the next best approach in a similar context or the current work is evaluating the approach.
  4. If a reviewer remarks that some prior work has tackled a similar problem, then the reviewer should provide references to such prior work.
    This applies to novelty aspect of the work.
  5. If a reviewer has feedback about specific claims made in the paper, then, in the review, the reviewer should quote snippets from the paper that imply such claims along with her comments. Specifically, when the feedback is negative.
  6. If a reviewer believes a specific contribution in a paper can/should be improved, then he should provide one or two ways of improving the contribution.
  7. If a reviewer wants to expose her student to the reviewing process, then both she and her student should review the paper and submit their reviews as two different reviews (by a reviewer and a sub-reviewer). In such cases, the reviewer should review her student’s review with her student before submitting the reviews. She should not delegate the reviewing task to her student when she is the reviewer.

If you have additional guidelines, then please suggest them by leaving a comment :)

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade