scite Before You Cite

What science can learn from a 146-year old legal practice

When I first heard the term “Shepardizing” from a lawyer friend of mine, I ignored it because I had no idea what it was and wrongly assumed it was irrelevant to what we are working on at scite. Indeed, it sounds like something that has to do with corralling sheep or some kind of meat pie, not scientific publishing. It’s safe to say that many scientists, if not most, don’t know the term. Case in point, none of us knew about it when we started scite. So what is Shepardizing and what can science learn from it?

Shepardizing

Shepardizing is a term named after Frank Shepard who, in 1873, introduced a process by which, “An individual checking a citation..will be able to find out various information, such as how often the opinion has been followed in later cases and whether a particular case has been overruled or modified.” In short, it allows lawyers to search and cite reliable law without needing to read every court case and opinion ever referencing it. Today, software has made it easy to check to see if a case has been overturned or not with a simple search.

And while Shepardizing has become a standard practice in law, there’s been no parallel in scientific research despite the idea being discussed as early as the 1920’s as the creator of the Citation Index, Eugene Garfield, describes in this video!

The fact that such a system like this doesn’t exist in science has had serious negative effects in research as well as society. For example, researchers can selectively use supporting citations to establish their work as fact and to promote their findings. Steven Greenberg, a researcher at Harvard Medical School, highlights how this occurs in medical research by showing that researchers use citations to create unfounded authority in order to get grants and further one’s own research agenda. He analyzes the research literature behind the claim “that β amyloid and its precursors are abnormally and specifically present in inclusion body myositis muscle fibres.” By looking at hundreds of papers he identifies that although critical studies exist in this network they are selectively ignored thereby creating a biased narrative.

Such biased citation pattern highlights the key issue: there is no easy way to tell if scientific report has been supported or contradicted. There is no Shepardizing-like mechanism in science. Further evidence of this is seen in the fact that researchers continue to cite work, even after it is retracted! The reason for this is that scientists are drowning in information. Indeed, in order to understand if a scientific report cited 100 times has been supported or contradicted one would need to read 100 papers. This is simply not practical. In law, they would find this information by doing a simple search.

sciting for science!

We’re solving this problem with the introduction of scite, a new way for researchers to automatically identify if a scientific report has been supported or contradicted and more broadly to see how something has been cited, not just how many times.