How posting a preprint made our paper better

Hello! My name is Rich. I’m a first-year PhD student. My first paper was just published: “Tracking the popularity and outcomes of all bioRxiv preprints.” There’s a Twitter thread if you want a rundown of the paper itself, but the part I want to share today is about the experience of going through the submission and review process, and how it was greatly improved by first posting the paper as a preprint.

We posted the manuscript to bioRxiv on 13 Jan 2019 and submitted the paper to eLife later that month. It was sent out for review in early February. On March 1, biOverlay posted two peer reviews of the preprint. biOverlay is a unique website r̶u̶n̶ “instigated” by Casey Greene at the University of Pennsylvania: It’s maintained by a cadre of (volunteer) editors who solicit peer reviews of interesting preprints and post them online. Their feedback about our preprint was thoughtful and precise, and we planned to incorporate some of it into the next revision of the paper, wherever it ended up.

On March 5, we got the official eLife decision letter back requesting revisions. The letter linked to the biOverlay review introduction and quoted from it extensively. The editor wrote, “The most important comments in that review are included in the points below. If there are additional comments from biOverlay that you wish to address in the revision please highlight these in your author response.” (The journal attaches its decision letter to each published paper.)

The effect of posting a preprint would have been similar even if the timing had not been so tidy: We got five reviews for our paper (three from the journal, two from biOverlay) with helpful suggestions. The biOverlay review was edited later to incorporate links to other research based on our data, and their website’s hypothes.is integration allowed people to annotate the biOverlay reviews with suggestions and feedback. Our paper was written up by preLights, a site that highlights preprints of “particular interest to the biological community.” There were also multiple blog posts and more than 1,300 tweets about the preprint — a lot of these just said “Cool!” (thank you) or “Kinda revolting” (point taken), but there was a lot of interesting conversation in between. An added benefit was that the preprint was cited by several papers, which makes me feel nice but also helped inform the research in those studies.

If we hadn’t posted a preprint, those tweets wouldn’t have happened, the biOverlay reviews wouldn’t have happened, and we would have received much less feedback about how to improve our work. Granted, my experience was exceptional — not every preprint gets this much help from the community, not every journal will incorporate unsolicited external reviews into their editorial process, and not everyone’s advisors will be as enthusiastic about preprints and community feedback. But our experience with bioRxiv, biOverlay and eLife helps illustrate the practical implications of the more hand-wavey benefits of openness and transparency: People saw our work before it was engraved in the permanent record. They got to tell us what they liked and, more importantly, highlight areas that needed some more work. Why wouldn’t we want to do that? It was a little unnerving to face that kind of scrutiny, but that scrutiny’s going to show up either way — and it was a lot better to receive it while there was still time to make improvements.

(Full disclosure: I think preprints are good, and I am an ambassador for ASAPbio, a nonprofit that advocates for transparency and innovation in life science communication. If you’re interested in more information about preprints, ASAPbio has an overwhelming amount of information about them, and you can feel free to email me any time at rabdill@umn.edu.)