Should we cite blog posts during peer review?
No.
Oh, wait, you want more? OK. There was a discussion this morning on twitter about whether or not citing a blog post to support a claim during the reply to reviewers was a a good point. My reply is a very firm “no, under no circumstances”, and I will explain briefly why.
First, do not bring a blog post to a primary literature fight. If the reviewer manages to find a citation supporting their claim, and you offer a blog post in response, the tradition of how we weigh evidence in academia are playing against you.
Second, blog posts are not tools of scientific discourse. Well, they can be. But not always. Let me clarify. Blogs do absolutely have value as an outreach tool within our community, and as a way to discuss the way we do science, that is, the practices of being a scientist — what I write here fall within this category. Blogs can be used for outreach to non-professional-scientists, and have been for a long while. And blogs can be used as a scratchpad for novel ideas — a few of my papers started as blog posts, and this includes primary research (and interestingly enough, this includes a paper I wrote to show that a claim that both reviewers made during the review of another manuscript was wrong, which started as a blog post; we delayed the resubmission of the first manuscript by two years until the other paper was on-line and usable to show why we were right). But what blog posts are not are a final product that we can consider as having the same status as a manuscript. This is where the difference between blog posts and preprints lays: preprints are intended as the raw material which we put in the editorial sausage making machine. Blog posts are (rare exceptions aside) not.
Third, blog posts are highly subjective. This is a strength, especially when it comes to discussing the way we behave in our scientific community. But this is a weakness when it comes to establishing the scientific record. This is why, although a blogger myself, I cringe when I see a blog post cited in a paper. This is an easy way to pick information that matches what you want to say, without the necessary objectivity that we expect from primary literature. As anecdotal evidence, I offer the fact that most of the citations to blog posts I found where in “Opinion” type of papers, for which we have lower standards of objectivity (hence the name).
Finally, responses to reviewers are not a discussion. This is something on which I have strong opinions, but I will try to be brief. As soon as one side in the discussion has all the burden of proof (this is us-as-authors), and the other side has an assumption of being right (this is us-as-reviewers), this is not a friendly chat. This is a process in which what is said is weighed against the arguments of the opposing party, based in no small part on the validity of what is used to support each side’s claim. The real arbitrator of this process is the editor. Blog posts are fine to present a point of view when discussing with colleagues; but as an editor of reviewer, I would not consider them “acceptable evidence”. This would not disqualify the manuscript; but the balance of peer-review is easily tipped, and anything which does not contribute to tipping it in your favor is a waste of ink.
So there you have it. In my mind, the goal of replying to the reviewers comments is to convince the editors that you have done a good job, and that this job relies on the accepted best practices in the field. Citing a blog post just looks sketchy, regardless of the reputation of the blog, or the person writing it (especially so, because although appeal to high number of page views is not a logical fallacy, it should be). If you cannot find primary literature to support a claim, try to make a case for it. But I think that citing a blog post has a strong risk of being viewed as a weak case, and you cannot afford this during the review process.
