Peer Review in 5 points
Peer Review Week has brought many thoughtful posts about the past, present, and future of peer review. As a believer in the correct framing of ideas in order to discuss them, I thought I would supply 5 quick(ish)-fire thoughts in bullet-point format, for your consideration.
- Peer review is not a system, nor a movement, nor a thing that can be “fixed” or “broken”. I think of peer review as an activity, currently performed by humans, that can be done well or done poorly, and everything in between. Therefore, attitudes, biases, workloads, cultural norms, personal sensitivities, time, etc. are all more important and, in fact, are usually the true discussion topics when we have these conversations.
- Peer review, as far as I am concerned, means review. by. peers. Therefore, it means pre-publication peer review. It means post-publication peer review. It means open peer review. It means peer review where reviewers sign their names. It means annotations. It means commentary. It means any way we can think of which allows peers to review something. The same person is able make the same comment about a paper in a pre-publication review process or an online annotation (but I am not saying the outcome or experience is the same). Much discussion depends on which specific type you are talking about, so we must always be clear about this. But they are all peer review if they involve peers reviewing something.
- Open Access journals like Collabra: Psychology do not perform a “light” (or even “lite”) version of peer review just because they focus on more objective and methodological criteria. Nor do more selective, impact-seeking journals necessarily perform more rigorous or “heavy” peer review. This scale is not about what is better, or goes deeper, but is usually about objectivity and subjectivity, impact-seeking, and topic curation. Some selective journals reject papers because they feel they have published too much on [Insert Topic X] recently. That is topic curation. Some selective journals accept papers because they know [Insert Topic Y] is a popular topic which will get media exposure. That is impact-seeking. This is a different type of peer review to one solely focused on rigor, not a better or heavier type of peer review.
- Some journals do not seem to be performing any kind of peer review and, if they are OA journals, seem only interested in receiving your Article Processing Charge. I’ll keep it brief: their existence is the fault of nefarious opportunists, not other, honest, OA journals!
- Open peer review, such as the option at Collabra: Psychology whereby the comments of the peer review process are published alongside an accepted article, is interesting and good because it helps you decide how much you trust the peer-reviewed article, and gives you great background regarding the article’s development. All the other arguments around open peer review (good; bad; dangerous, etc.) are still extremely important, but are not actually about open peer review. They are about the weird, wonderful, hierarchical world of ambitious, sensitive humans in the academic and research infrastructure. Interestingly, as an option at submission at our journal, 82% of authors have chosen this, so far.
One last note: Collabra: Psychology is the evolution of Collabra, the open access journal published by the University of California Press that shares the actual value generated by the publishing operation with everyone who works on it. While Collabra: Psychology editors and reviewers do not attempt to predict a submission’s impact to the field, they will check for rigorously and transparently conducted, statistically sound, adequately powered, and fairly analyzed research worthy of inclusion in the scholarly record. This focus on more objective acceptance criteria should not be mistaken for a light version of peer review—the bar is set high!
— Dan Morgan, Publisher, Collabra: Psychology, UC Press
