Deny deniers the energy they feed off!

Scientific papers continue to be cited long after being retracted. That needs to change.

blob=paper, size=#citations, distance=time, color=journal

Imagine you’re a scientist. You published a paper in a reputable journal a couple of years ago, and that paper has made an impact in your field of study. It has shaped your own research and inspired others to pursue similar lines of investigation. Thanks to that paper, you’re enjoying a certain status among your peers. Now you find out that one of the works you cited in that paper has just been retracted. Whether it was an honest mistake, total fabrication or something in between, that work has now been flagged and it doesn’t stand up to scientific scrutiny anymore.

What do you do?

Modern science is a group effort. It’s slow, it’s complicated, it often involves multiple disciplines and it’s almost always expensive — all excellent reasons for researchers to team up to share the burden and check each other’s equations. A 2015 paper in Physical Review Letters had a record-breaking 5,154 co-authors! That’s a bit extreme, of course, but there’s no question that across many fields today, single-authorship is shrinking and multi-authorship is growing.

And science is built on top of science. Standing on the shoulders of giants, as Google Scholar makes clear on its home page. Researchers read up on prior work before starting their own lines of inquiry, often replicating past experiments to lay a solid foundation for their own research. They cite their references scrupulously, and peer-review is the norm before anythings gets published.

All of this to say that scientists form a giant altruistic community. There are turf considerations and funding to compete for, as in any other walk of life, but by and large, scientists are doing what they do because they’re motivated by the greater good, and collaboration is in their DNA.

So when a paper is retracted, potentially jeopardizing everything that came after it, the scientific community goes into self-correction mode and pulls all the papers that, at one point or another, directly or indirectly, have made a reference to that work, right? Until further notice? Not exactly.

For one thing, the word that a paper has been retracted rarely makes the news. Unless, of course, that paper is this piece of malfeasance and your name is Andrew Wakefield. Most of the time though, the original publisher simply publishes a retraction notice, slaps a ‘retracted’ watermark on the online version of the paper and notifies the online databases. But the original paper still lives on in print (which is where you might have read it in the first place), and you don’t receive an alert that one of your citations has just imploded.

Which means that you might be totally in the dark.

Can your own work survive without that one citation? In the vast majority of cases, it will. But why keep the citation in when it’s likely to cast a shadow on your work? And in those few situations where the retraction shakes the very foundations upon which your house is built, you’re going to want to address the situation head on — and sooner rather than later.

It’s no secret that science is under attack. The outstanding team at Retraction Watch catalogs around 600 new cases every year. That’s not necessarily a bad thing — after all, a high number of retractions is a healthy sign that science is actively correcting itself. And when you compare that figure to the volume of scientific papers out there (2.5 million new papers every year, published by more than 35,000 journals worldwide), it’s a very small percentage. But a good half of all retractions are due to misconduct, and political interests are quick to pounce and use those cases to tar an entire field of study. In today’s out-of-control news cycle, allowing a single retraction to have a ripple effect on the credibility of future research is simply unacceptable.

So, what do we do about it?

In a recent request for information, the Defense Advanced Research Projects Agency (DARPA) recognized the problem and issued the following grim assessment:

Publications today lack mechanisms for rapidly incorporating dynamic changes in the literature, such as retractions, replications, or new findings, which presumably should alter a user’s confidence in a specific study’s conclusions.

The agency is valiantly inviting researchers to help them devise some automated system to assign confidence levels to research results. Beyond p-values, impact factors, h-indices and altmetrics, they’re looking for some combination of natural language processing, sentiment analysis, network analytics and machine learning to quickly assess a research claim’s “reliability, reproducibility and robustness.”

This is commendable and definitely worth exploring, but it’s hard to imagine that the solution won’t involve some seriously low-tech grunt work: namely, alerting all authors who have cited the retracted work, directly or indirectly, and asking them to formally evaluate the impact of that retraction on their own work. We can debate how ‘formal’ that process ought to be (should all affected papers be shelved until they’ve been cleared?), but there’s little doubt that it needs to happen.

This is no small matter.

As the animation at the top of the article showed, one stem cell paper published in 2005 and retracted in 2010 has been cited 667 times — so far. Nearly half of those citations occurred after the retraction was made official. Here we are, in 2017, seven years after its retraction, and authors continue to refer to it as if nothing happened. Nobody knows the extent to which the mistakes in that paper have affected any of those papers downstream.

And the animation at the top only accounts for direct citations. Each one of those 667 papers were, in turn, cited by other papers, and so on and so forth. See how the ripple effect from that one retraction ends up affecting more than 33,000 papers:

2nd-degree impact from one single retraction

Science is a powerful thing. And part of what makes it powerful is how interconnected everything is. But that interdependence is also what makes it vulnerable.

It doesn’t have to be. By staying on top of retractions and jumping at the opportunity to reassess their work when a source has been tainted, scientists will deny science deniers the energy they feed off, and go a long way to improve the standing of their own research.