Assessing Science: A Perspective From The Cheap Seats
James Heathers

I enjoyed reading your thoughts, and I think I’ve made my feelings about peer review clear in my posts.

One of my biggest frustrations that I haven’t discussed before is the limited amount of interaction between authors and reviewers. Like, what year is it, are we sending letters by horseback? You get your reviewer comments, submit your response, then have to wait a month to get your next round of comments, which often still contain the same misunderstandings that you just addressed in your rebuttal; guess you’ll have to wait another month to see if your reviewers can understand basic concepts, tough luck.

And from the perspective of a reviewer, sometimes I have serious concerns about a paper, but I’m not completely sure I’m interpreting the paper correctly and if my concerns are valid. So I’ll submit a response with my concerns and my review will get labeled by the editor as “major revisions necessary” or some bullshit, and then I’ll never hear from the authors again (I guess they didn’t submit a revision or the paper was rejected), so I’ll never know if the authors were making the mistakes I thought they were or if it was a misunderstanding.

Can’t we just get all the reviewers and authors into some sort of anonymous group forum/email chain/skype call/anything??? Why can’t I talk to the other reviewers? Maybe there’s a mathematical component of the paper that I wish I could ask someone with more expertise about. Should I have to send the question to a mathematician I know, and would that violate some sort of confidentiality agreement of the peer review process? Why can’t I just ask the other reviewers if they feel comfortable about a part of the paper I don’t understand and see what they say?


Another thing, I don’t agree that we can’t kill prepublication peer review. What’s wrong with the postpublication review model of F1000Research? I know you believe in preprints, so why not just force people to make their research a preprint awaiting formal peer review?

I see some huge benefits of having every paper public awaiting peer review. This allows reviewers to self-select themselves, and I think we can all agree self-selected reviewers are better than random reviewers. And if we can get a system with huge rewards for performing peer review, scientists will be chomping at the bit to review papers that they are qualified to review.

On a somewhat unrelated note, I hope you saw this news since you predicted this on your podcast:

If you have any other predictions you’d like to share let me know.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.