Nov 17, 2020
Checking for retractions & other quality checks on your manuscript before journal submissions — some free tools scite, Zotero, Pubpeer, Scholarcy, libkey and more
In my last post, I talked about the state of retractions in the literature and how so much of the literature includes authors who probably unknowingly cites retracted works
As journals (possibly even peer reviewers) are increasingly screening citations for retractions, it might be also a good idea to do your own screening before you submit your manuscript to avoid embarrassment. It is one thing to cite on purpose a retracted work , another to be unaware of what you are doing.
Below I detail two types of tools you can use — firstly pre-submission manuscript health check tools and secondly Reference Managers with useful features that check citations for retractions and other useful signals.
As an author — you can use pre-submission “manuscript health-checks”
One thing some authors like to do before submitting their manuscript to journals is to run checks via iThenticate, to check for plagiarism, I personally have quite a lot of views on this use, but I shall leave it to another essay to discuss this.
But I don’t believe this handles retraction checks. Instead a great free check to run is scite’s reference checker.
The idea here is you can upload any paper, probably a manuscript that has not been published and scite will produce a report , similar to the scite entries you see for published papers.
While scite is more known for “smart citations” where they use machine learning to classifying citations into “mentioning”, “supporting” and “disputing” cites in this case, scite also checks for retractions or rather “editorial concerns” e.g. retractions from Crossref, Pubmed and their own algos (they no longer use Retraction Watch database)
The manuscript check, will parse and extract citations from your manuscript and check for retractions. Below shows one example.
Edit : a cool addition is the scite Chrome extension now works on Wikipedia references and yes it can detect retractions too.
Since you are already into checking manuscripts, there’s a third one you can try Scholarcy preprint healthcheck API
It extracts affiliations, highlights key findings, figures out study subjects, statistics , tries to identify sections like data availabilty, ethical compliance etc.
There is some overlap in features as the scite check as it also checks for retractions via scite and Europe PMC APIs. In terms of references it extracts and runs them against various whitelists or sources like DOAJ and Crossref, which I guess can be useful if you want to filter references by the reputation of the source.
The scite browser extension is also helpful, as the scite badge now warns you of editorial concerns.
What you can do as an author — use a Reference Manager that does autochecks for retractions & why Zotero is your best choice.
For many researchers, their reference manager is pretty much the centre of their own personalized research universe, as such it makes a lot of sense to do such checks everytime they open they reference manager. But how many do that?
I first came across the idea of a reference manager, PubChase that used the Crossref API to check for retractions in 2017 and it blew my mind (I was still new to the idea of Scholarly graphs at the time).
Of course today, many reference managers are starting to check for retractions as well. As I write this in 2020 we are starting to see this feature become common for example, Readcube/Papers just added this feature in Nov 2020 and more will likely follow.
But recently, it has become harder and harder for me to recommend any other reference manager other than Zotero.
Why? Firstly the basics. Zotero does indeed check for retractions for papers in your Zotero library. It does it beautifully by not only warning you when you import the reference into Zotero
But also warning you when you try to cite it later using the plugin. Making it extremely hard to miss citing a retracted paper unknowingly.
All this is built-in natively in Zotero and it checks the retraction watch database.
Zotero plugins — Scite and PubPeer provide an even better radar
Zotero was one of the most reference managers to provide retraction checks using the excellent Retraction Watch Database. But with Zotero you can use two more free plugins — Scite and PubPeer to get an even better sense about the papers you are thinking of citing.
Why use them?
A recent paper entitled “Self-correction of science: a comparative study of negative citations and post-publication peer review”, attempted to study whether there was any correlation between number of comments in the post publication review systems PubPeer or contradicting cites (now renamed in scite as disputing cites) classified by scite between 2 sets of papers — papers that were eventually retracted and non-retracted papers.
Interestingly
“On the whole, these results show that negative (disputing) citations are not more frequent towards retracted or corrected articles”
yet
Looking at number of PubPeer comments look more promising, on average there seems to be more comments for retracted/corrected articles (particularly for specific author corpus that end of retracted).
The authors speculate that because PubPeer comments could be anonymous, commenters were more willing to query suspicious results, which often eventually snowballed into a retraction compared to openly doing a disputing cite in their published paper.
I would add on to speculate that if authors “smell a rat” with a paper, they would tend to avoid citing the paper altogether rather than give a disputing cite and in fact a disputing cite would usually be honestly given because the citing author felt there were reasons for the differences that were not fraud related.
scite on Twitter responded that while number of disputing cites are low, they are still higher in frequency than Pubpeer comments , so it might still be useful . I also wonder if post publication comments from Publons might be worth adding.
I’m still trying to puzzle out what these findings mean, part of the problem with the research design is I *think* it does not differentiate between comments made be retraction and after and the same for disputing cites which might affect cause and effect (though it is probably fine with correlations).
In any case, as a researcher it does seem useful to be able to quickly look at these signals around a paper you just imported into your reference library.
And indeed with the two Zotero plugins for PubPeer and scite respectively you can!
Below shows the PubPeer plugin installed and you can see a new column for PubPeer appear (with icon and number of comments) as well as a tab that allows you to link to the PubPeer comment.
This allows you to tell at a glance with papers seems to have particularly high number of comments, which might indicate something “interesting” going on.
Similarly you can install the brand new scite Zotero plugin to see the number of “mentioning”, “supporting” and “disputing” cites as per classified by scite Machine learning algothrims.
And you can always go to the actual scite report, by right clicking to display context menu….
Even if disputing cites do not reliably indicate papers that are likely to be retracted, at the very least this would help you also see which papers you are planning to cite that are quite controversial.
Other ways (edit April 2021)
More and more systems are starting to try to do retraction checks.
These include Lean Library browser extension which is mainly used to help with access and has now partnered with scite. and ThirdIron Libkey services (e.g. Libkey Nomad, Browzine etc) — (if your library has a subscription you should try this)
Conclusion
Zotero has always been a leading free, open source reference manager, but recently it has been really firing on all cylinders as more and more interesting plugins are developed for it.
Between it’s native build-in retraction checks via retraction watch database, PubPeer plugins and scite plugins that allow you to potentially catch signals that the paper you are planning to cite is “interesting”, you have quite the radar….
And if I am not wrong, Zotero being open source is going to be further supported by more plugins and features, not least is the WikiCite addon for Zotero with citation graph support, which has been funded and is scheduled for release in April 2021.
