Why I don’t send papers to journals: A case study
I’m already on record that I don’t send my work to journals, but I can’t control what my coauthors do. Unfortunately, one of my recent papers did get submitted to a journal, but at this point I think my coauthors may be regretting that decision.
The paper in question is the famous pizzagate story. Basically a researcher admitted to questionable research practices in a blog post and we were intrigued enough to take a look at the resulting papers. We immediately identified numerous granularity inconsistencies, then went on to find more problems, a lot more.
We attempted to contact the authors to understand the problems, but we were told to go get our own data. The question then became what to do with our findings. Do we blog about them? Send a letter to a journal?
Since leaving academia I’ve been posting my scholarly work as preprints, and recently learned LaTeX to get some of that journal-added value, so I was inclined to post our analysis as a preprint. Nick Brown had a good experience with his granularity preprint, having been picked up by the media (and getting me involved with the project), so it wasn’t too hard to get my coauthors on board with posting our work to PeerJ Preprints. However, they still wanted to send it to a “real” journal.
We opted to send it to BMC Nutrition, given that they published one of the pizza papers and have several open science friendly practices such as open peer reviews and open data (supposedly). Our submission was contingent on them waiving all publication fees, since our work was unfunded and they were partly responsible for pizzagate.
As usual it took a while for the paper to get reviewed, and once it finally got reviewed no mistakes were found and no changes to the manuscript were requested, consistent with my previous experiences with peer review.
And this is when problems began to arise. In the manuscript we claim that Cornell has refused to share their data, which was true at the time, but by the time the paper got accepted Cornell had released their precious data. The acceptance letter asked if there are any developments we should notify them about. Umm, yes, it just so happens that over a period of a couple months things happen.
Of course we notified them of the data release, but they didn’t seem to care and proceeded to work on getting the proofs ready. Okay, cool, so I guess the paper’s getting published?
Not so fast, for some reason they decided to see if Cornell wanted to respond to our paper. That’s…interesting, I wonder what Cornell will say.
Apparently not much, because the proofs got made.
As is customary, the journal is allowed to take months to send out a paper for review but when it comes to proofs the authors must respond IMMEDIATELY. Well, we took a look at the proofs and noticed all the table numbering is wrong, links are broken, the black and white text makes some details hard to see…
Normally I would complain about publisher-added value, but we aren’t paying the journal so I guess we’re getting what we paid for. I’m not sure how long it will take to get these things sorted out and I have no idea when the paper is getting published.
Okay, so what’s the big deal? The work has been available since January as a preprint so why does it matter when the journal version comes out?
Well, once Cornell released their data in April I went ahead and reanalyzed it and wrote up the analysis. I wasn’t sure when to make the analysis public given we had a paper about to be published that claims we don’t even have access to the data. It would be weird if I made the analysis public before the paper was published, right?
After getting contacted by more journalists about this story I decided I couldn’t hold back my reanalysis any longer, and sent it to PeerJ Preprints. It got posted within a few hours, immediately indexed by PrePubMed, and blogged by Andrew Gelman the next day.
So what’s going to happen with the BMC Nutrition paper? I don’t really care. I don’t consider it the version of record. On my CV I will still link to the preprint, which contains reader feedback in addition to 60 curated links related to the preprint.
Journal publishing can no longer keep up with the pace of scientific research (did it ever?), and if it adds any value it certainly isn’t worth the effort and time. There’s just no reason to send papers to journals unless you are trying to impress a search committee or enjoy paying someone to give them the copyright to your work.