Cognitive Rationality, Systematic Bias and the Death of Science

There is a growing concern amongst Scientists about a worrying trend in Scientific methodology which, if unchecked, may lead to worsening outcomes of Scientific research. There is evidence of consistent failings in the Scientific method, across all fields, from cosmology to medicine in which Scientists, both accidentally — and sometimes deliberately — fail to follow the Scientific method, leading to incorrect research being published and circulated.

What Happens

It is often assumed that the results of a Scientific study are always reliable, as it was conducted using the Scientific method; unfortunately, this is not always the case. It is frequently seen in Science that a theory which has been presented, which at the time was peer reviewed and widely accepted, is later refuted by subsequent research. Many of these cases appear in the news, and have far-reaching consequences. These mistakes can be found across a broad range of Scientific research, from climate change, which often has conflicting theories, to genetically modified foods. Both areas of Science spark much media attention, and often cause disputes between Scientists.

Probability that a research finding is true as a function of the Pre-Study Odds for Various Levels of Bias, u

On top of the variety of the fields in which these failures can be found, the extent of this misinformation also widely varies from entirely incorrect theories to relatively minor issues. A respected senior psychologist published a study in a leading journal, in which he claimed to have found evidence of precognition. That claim was ridiculous, since it is clearly impossible to “see” events in the future. As an example of a smaller issue, an investigation in 2012 showed that half of published neuroScience papers contain at least one basic statistical error.

This is not a recent problem; examples of this can be found throughout history, from early Greek Science through to the modern day. For centuries the geocentric theory that the earth was at the centre of the universe, and that the sun, planets, and all the stars, orbited it, was held to be Scientific truth, and was not challenged until Copernicus published his book De revolutionibus orbium coelestium in 1543, and was not disproven until as late as 1687, when Isaac Newton devised The Law of Universal Gravitation. Early descriptions of the geocentric model can be found in 4th century BC Ancient Greek manuscripts. The geocentric model of the universe prevailed for most of modern civilisation, and yet was completely incorrect.

The problem with the geocentric theory was not so much its falsity, as much as that during the time it was held to be true, all Dark Age astronomy was based on a false construct. Thus, significant mistakes were made within observational astronomy. The repercussions of basing research on incorrect findings can be seen across Science. False theories in theoretical Science are problems for Science but in the field of medical research it is a very serious problem.

The Scientific method used today is not the best method of learning about the world, but it is better than any method used before, and better than that used in Ancient Greece. In fact, before Ibn al-Haytham wrote The Book of Optics (which contained the beginnings of ideas such as Occam’s Razor), it was not common practice for Scientists to test their hypotheses at all. Whilst the Science that we practice today may be better, it is still flawed, and there are major issues facing the Scientific community.

Why Scientific Mistakes are harmful

The impacts of false Science range from small issues of public misunderstanding to ones of much graver consequence.

An instance of public misunderstanding occurred in 2008, when there were widespread concern about CERN’s Large Hadron Collider. There were fears that the Collider would create a black hole and destroy the planet. These fears were (predictably), totally baseless: high energy conditions similar to those in a Black Hole were being created, but for nanoseconds. The energy required to create and sustain a black hole is far beyond that which could ever be created at CERN.

However, it is within medicine that the consequences of poor or false research pose the greatest threat; from general health advice — such as to drink lots of water, when in fact this can be fatal — to much more specific concerns. For most people, whether or not String Theory is correct could not matter or impact their lives less. However if a drug they are prescribed causes them ill health, or worse, then it is a far more serious problem. People may misunderstand experiments like those conducted at CERN, but when major controversies occur in medicine, people lose faith in the profession, and cease to trust their doctors.

An example was the drug Thalidomide which was marketed as a completely safe mild sleeping pill, and used throughout the world, with few noticeable negative effects. However, in 1960, many geneticists began to notice an ‘epidemic’ of children born with missing or malformed limbs, and it was suggested in 1961 that this was because their mothers had taken Thalidomide. This was eventually proven, and there was a seismic change in the process for new drug development. All new drugs that entered the market had to go through a long period of testing. This positive benefit did little to alleviate a general deep public mistrust of new drugs and medicine as a whole.

Another more recent example is the MMR vaccine, widely used to prevent Measles, Mumps and Rubella. In 1998, [Dr] Andrew Wakefield published a false paper in The Lancet claiming to show that the vaccine caused Autism. This lead to a rapid decline in the usage of the vaccine, which in turn created a dangerous outbreak of measles. There were needless deaths. Wakefield was shown to have manipulated evidence deliberately, and his paper’s claim was fully retracted from The Lancet in 2010.

The relationship between patients and doctors is one based significantly on trust. Both the cases above massively undermine this. Doctors do make mistakes, but only rarely, and it is those few mistakes that make the news. Thus, people are only exposed to these mistakes, which leads them to believe that medicine is much less reliable than it is in reality.

Problems thus arise when people no longer trust doctors to make the right decisions. They are less likely to follow the good advice that they are given, and this could have drastic effects on their health.

When people mistrust medicine, and by extension the whole of Science, this has wider reaching consequences. The government is a key source of funding for Science, and the government, ultimately, serves the will of the people. A loss of trust in Science may affect this funding. This means that good Science may not get enough funding, resulting in a slowing of Scientific progress as a whole.

While the public may feel skeptical about Science, huge amounts of taxpayers’ money is still being spent on research, some of which is potentially flawed. In 2008, research and development spend amounted to 2.3% of GDP for the OECD as a whole. In 2012 R&D spend totalled a staggering figure of 59 billion dollars.

As I will show later, even being charitable, if the “Scientific 5% rule” is applied, then at least $3 billion is being wasted — and in reality much more — this would be enough to build more than 150 secondary schools in the UK.

Why it happens

There are multiple reasons that a Scientific study could produce incorrect results. These reasons fall into two categories: accidental errors, normally due to statistical mistakes or cognitive biases; and deliberate manipulations of the experiment by the researchers in order to produce results they want.

Scientists do not always have completely pure motives; they are often influenced by other factors, aside from simply a desire to find the truth. As well as accidental failings, Scientists and researchers can sometimes deliberately manipulate results in order to produce an outcome that they want.

Scientists have two fundamental requirements which can be sources of pressure to them, and which could lead them to manipulate the results of their studies. The first is the need for funding for research, and the second is a requirement to publish regularly and in respected journals in order to be recognised as influential by other Scientists.

According to an article in The New York Times: “To survive professionally, Scientists feel the need to publish as many papers as possible, and to get them into high-profile journals. And sometimes they cut corners or even commit misconduct to get there.” The majority of Scientists aim to publish important research that produces significant advances in their field. A few may be more inclined to manipulate evidence, at worst, whether or not a particular drug has negative side effects.

Scientists can also be influenced by economic factors. Most medical research is funded by private drug companies, to develop new drugs for the market, thus, researchers have an incentive to discard evidence that a particular drug may not be safe in order to continue receiving funding.

This happened in 1999, when the company Merck introduced the drug Rofecoxib, designed to decrease pain due to osteoarthritis. In the early development of Rofecoxib, concerns had been raised by Scientists within Merck that the drug may increase the likeliness of cardiovascular disease. In 1996, Merck sponsored a study which examined these concerns, and when the study showed that it was likely that the drug had these side effects, the Scientists were asked by officials at Merck to “soften” the result in order to make it look less significant. The drug was then put onto the market, and while its dangers were later revealed, it had already caused nearly 30,000 people to suffer from heart attacks.

As well as deliberate manipulations of research papers, Scientists also produce false research accidentally. There are three primary reasons for these mistakes: first, a lack of repetition of experiments, second, cognitive biases within the Scientists conducting the experiment, and third, simple statistical mistakes that account for large number of incorrect findings.

The first problem stems from a lack of replication. While in schools we are taught the importance of repeating experiments, to improve the reliability of our results, in labs across the world. Scientists do not always do this, which is a key reason for many errors. A single Scientist or research group will have an hypothesis, and when an experiment confirms this hypothesis, or has an extraordinary outcome, the researcher(s) are likely to want to publish immediately rather than repeat an experiment. This is a problem as once an experimental result is published, the assumption is that it has been conducted correctly, and that the result can be trusted, particularly if published in a respected journal.

In addition not all findings are subject to the rigorous testing of repetition. Journal editors are eager for new and exiting discoveries, and thus are significantly less interested in scientific decorum. Most Scientists also want to make a name for themselves independently, and would much rather work on new research that is likely to add to their standing in the Scientific community than repeat someone else’s experiment. This is particularly prevalent in younger researchers, as it is often felt that repeating their superiors’ experiments will be taken as disrespectful, and will lead to a slower career advance.

The second reason for accidental failings is inbuilt biases. There are many biases that affect researchers. Three in particular have large effects on the outcomes of research: publication bias, conformation bias, and experimenter bias. These three biases are key in understanding the problems faced by the Scientific community today.

Publication bias, according to The British Medical Journal, is

“a well known phenomenon in clinical literature, in which positive results have a better chance of being published, are published earlier, and are published in journals with higher impact factors.”

Simply put, publication bias is the tendency for researchers and journal editors to publish positive results instead of results that are neutral or conform to the “null hypothesis”. This is particularly serious because one of the key advantages of the Scientific method is that proving something to be false, or disproving a hypothesis, is just as valuable as proving something to be true. This bias goes against this fundamental element; the dangers of this are clear:

“Simply put, when the research that is readily available differs in its results from the results of all the research that has been done in an area, readers and reviewers of that research are in danger of drawing the wrong conclusion about what that body of research shows. In some cases this can have dramatic consequences, as when an ineffective or dangerous treatment is falsely viewed as safe and effective.”

Another form of publication bias occurs when leading Scientific journals prefer to publish significant research that will improve the standing of the journal. Journals, especially online ones, in order to increase their content — sensational or otherwise — have two choices: they can work harder to publicise themselves, or they can lower the quality threshold. It should hardly surprise us that many choose the path of least resistance and lower the quality threshold.

On top of that, publishers and editors of such journals are in fact looking for unusual — or better — sensational findings, in order to raise the status of their journal. They are more likely to accept flawed research that makes an astonishing claim, without fully checking it, than to accept flawless studies confirming that present theories seem correct.

Another crucial factor here is conformation bias, the tendency selectively to favour pieces of information which conform to researchers’ theories or hypotheses. Scientists often subconsciously assign more weight to a particular claim than another, because they themselves believe that claim to be true. It manifests itself in many ways: for example, Scientists may discard hypotheses because they does not “fit” with what they feel is correct. Perhaps the most important manifestation of this is where Scientists create a hypothesis, and then rather than — as the Scientific method suggests — try to disprove it, they design experiments and look for evidence that will confirm it.

The third significant bias is the experimenter bias, which is the general term for biases in which a Scientist’s inbuilt expectation of a particular result can lead them to subconsciously affecting the outcome of an experiment.

There are seven stages of an experiment in which this bias can occur:

1. In reading-up on the field,

2. In specifying and selecting the study sample,

3. In executing the experimental manoeuvre (or exposure),

4. In measuring exposures and outcomes,

5. In analysing the data,

6. In interpreting the analysis, and

7. In publishing the results.

Perhaps the most prevalent source of failings in Scientific research however is statistical errors. The generally accepted convention in Science is that, when testing a hypothesis, if the probability that positive data could have occurred by chance is less than 5%, then the trial study is deemed statistically significant. What this means is that one result in 20 will be a so called “false positive”, or type 1 error. Scientists recognise two types of statistical error, type 1 errors, and type 2 errors. Type 1 errors occur 5% of the time, when an incorrect hypothesis is thought to be correct; type 2 errors, or “false negatives”, are where a correct hypothesis is perceived as being incorrect. The power of a study is the measure of how likely a false negative is to occur, with a power of 0.5 meaning that a false negative will occur half the time. A power of 0.8 is considered acceptable in the majority of cases, however it is rarely met.

While it is often thought that the 5% of false positives account for the statistical failings of Science, Dr John Ioannidis, in 2005, published an essay which claimed to show that “most claimed research findings are false”. As discussed earlier, he stated that a primary problem was a lack of repetition of experiments, which allows statistical flaws that are built into the system to manifest themselves. Ioannidis said that the idea of statistical significance ignores the probability of type 2 errors, as well as the extreme unlikeliness of any particular hypothesis being correct. He used Bayesian reasoning in order to validate his claim. This example from The Economist illustrates his point:

Consider 1,000 hypotheses being tested, of which just 100 are true (see chart). Studies with a power of 0.8 will find 80 of them, missing 20 because of false negatives. Of the 900 hypotheses that are wrong, 5% — that is, 45 of them — will look right because of type 1 errors. Add the false positives to the 80 true positives and you have 125 positive results, fully a third of which are specious. If you dropped the statistical power from 0.8 to 0.4, which would seem realistic for many fields, you would still have 45 false positives but only 40 true positives. More than half your positive results would be wrong.

Illustration of the dangers of false positives

How to fix it

These scientific problems are in many parts, and thus is not an easy issue to resolve as there is no one clear course of action to take. The first step would be to introduce Bayesian statistics as a norm for all studies, as well as the possibility of teaching rationality to Scientists in order to try to cancel out some of the biases that would otherwise affect them, as well as to alert Scientists to the danger and frequency of false positives. To tackle deliberate misinformation, researchers’ incentives need changing, and drastic alterations will have to take place with regard to the transparency of clinical studies.

In a world where the ‘problem of information’ has moved in rapidly from a lack to an abundance of it, and the key talent has moved from finding to filtering, that is a talent we need to further develop. We need to learn not to blindly trust information we are given, but to greet everything with a sceptical eye, to truly think about the information we are presented with. Throughout history, the most powerful driver of quality has been transparency, and if we manage to bring this transparency to Science, it will spark a new Scientific era of collaboration, communication, and success.

What is clear though, and has never changed, is that any human being developing an idea, study, or paper, will tend to overweight those sources and arguments that support their views. Indeed, this paper in itself is an example of the biases which it contains. It is clear that I have a position on this point, and have not emphasised the many hundreds of thousands of outstanding pieces of Scientific research that show exemplary neutrality. But, then again, they are not the ones that should worry us.

Work Consulted

All items are listed in are in the Chicago Manual of Style 16th Edition (note) format. All sources consulted during research are listed below in the same format, as well as original inspiration for the project.

John Ioannidis. “Why Most Published Research Findings Are False.” 2005 (n.d.). <http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124>.

Freedman, David H. “Lies, Damned Lies, and Medical Science.” The Atlantic, November 2010. <http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/.>

Boseley, Sarah, and health editor. “Prozac, Used by 40m People, Does Not Work Say Scientists.” The Guardian, February 26, 2008, sec. Society. <http://www.theguardian.com/society/2008/feb/26/mentalhealth.medicalresearch>

Oreskes, Naomi, 2007, “The Scientific consensus on climate change: How do we know we’re not wrong?” Climate Change: What It Means for Us, Our Children, and Our Grandchildren, edited by Joseph F. C. DiMento and Pamela Doughman, MIT Press, pp. 65–99.

Article Retraction from Journal Food and Chemical Toxicology <http://www.elsevier.com/about/press-releases/research-and-journals/elsevier-announces-article-retraction-from-journal-food-and-chemical-toxicology.>

French, Chris. “Precognition Studies and the Curse of the Failed Replications.” The Guardian, March 15, 2012, sec. Science. <http://www.theguardian.com/science/2012/mar/15/precognition-studies-curse-failed-replications.>

Nieuwenhuis, Sander, Birte U. Forstmann, and Eric-Jan Wagenmakers. “Erroneous Analyses of Interactions in NeuroScience: A Problem of Significance.” Nature NeuroScience 14, no. 9 (September 2011): 1105–1107. doi:10.1038/nn.2886.

Cessna, Abby. “Geocentric Model.” Universe Today. <http://www.universetoday.com/32607/geocentric-model/.>

Williams, James (2007) Do we know how Science works? A brief history of the Scientiﬁc method. School Science Review, 89 (327). pp. 119–124. ISSN 0036–6811

Reichenbach, Hans. The Rise of Scientific Philosophy. University of California Press, 1973.

Lehrer, Jonah. “The Truth Wears Off.” The New Yorker, December 13, 2010. <http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer?currentPage=all.>

Harrell, Eben. “Collider Triggers End-of-World Fears.” Time. <http://content.time.com/time/health/article/0,8599,1838947,00.html.>

Ballantyne, Coco. “Strange but True: Drinking Too Much Water Can Kill: Scientific American.” <http://www.scientificamerican.com/article.cfm?id=strange-but-true-drinking-too-much-water-can-kill.>

The Science Museum, Brought to Life, “Thalidomide” <http://www.sciencemuseum.org.uk/broughttolife/themes/controversies/thalidomide.aspx>

Choices, N. H. S. “MMR Vaccine ‘does Not Cause Autism’ — Health News — NHS Choices,” November 21, 2013. <http://www.nhs.uk/news/2007/January08/Pages/MMRvaccinedoesnotcauseautism.aspx.>

University of Berkeley, Understanding Science, “Who pays for Science?” <http://undsci.berkeley.edu/article/who_pays>

OECD Factbook 2011–2012: Economic, Environmental and Social Statistics, “Expenditure on Research and Development.” <http://www.oecd-ilibrary.org/sites/factbook-2011-en/08/01/01/index.html?itemId=/content/chapter/factbook-2011-68-en.>

James, Sebastian “Review of Education Capital.” <https://www.education.gov.uk/consultations/downloadableDocs/James%20Reviewpdf.pdf.>

Zimmer, Carl. “Rise in Scientific Journal Retractions Prompts Calls for Reform.” The New York Times, April 16, 2012, sec. Science. <http://www.nytimes.com/2012/04/17/science/rise-in-scientific-journal-retractions-prompts-calls-for-reform.html.>

Cohen, D. “Rosiglitazone: What Went Wrong?” BMJ 341, no. sep06 2 (September 6, 2010): c4848–c4848. doi:10.1136/bmj.c4848.

Krumholz, Harlan M., Joseph S. Ross, Amos H. Presler, and David S. Egilman. “What Have We Learnt from Vioxx?” BMJ 334, no. 7585 (January 20, 2007): 120–123. doi:10.1136/bmj.39024.487720.68.

“Trouble at the Lab.” The Economist, October 19, 2013. <http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble.>

Dubben, Hans-Hermann, and Hans-Peter Beck-Bornholdt. “Systematic Review of Publication Bias in Studies on Publication Bias.” BMJ 331, no. 7514 (August 20, 2005): 433–434. doi:10.1136/bmj.38478.497164.F7.

Rothstein, Hannah, A. J Sutton, and Michael Borenstein. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. Chichester, England; Hoboken, NJ: Wiley, 2005.

Plous, Scott. The Psychology of Judgment and Decision Making. New York: McGraw-Hill, 1993.

Nickerson, Raymond S. (1998), “Confirmation Bias; A Ubiquitous Phenomenon in Many Guises”, Review of General Psychology (Educational Publishing Foundation).

Wason, P. C. “On the Failure to Eliminate Hypotheses in a Conceptual Task.” Quarterly Journal of Experimental Psychology 12, no. 3 (1960): 129–140. doi:10.1080/17470216008416717.

Sackett, David. L. (1979). “Bias in analytic research”. Journal of Chronic Diseases. Vol. 32. pp. 51 to 63. (Pergamon Press Ltd 1979. Printed in Great Britain).

Cowles, Michael, and Caroline Davis. “On the Origins of the .05 Level of Statistical Significance.” American Psychologist 37, no. 5 (1982): 553–558. doi:10.1037/0003–066X.37.5.553.

“Power Analysis, Statistical Significance, & Effect Size.” <http://meera.snre.umich.edu/plan-an-evaluation/related-topics/power-analysis-statistical-significance-effect-size#significance.>

Bakker, Marjan, Annette van Dijk, and Jelte M. Wicherts. “The Rules of the Game Called Psychological Science.” Perspectives on Psychological Science 7, no. 6 (November 1, 2012): 543–554. doi:10.1177/1745691612459060.

“Trouble at the Lab.” The Economist, October 19, 2013. <http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble.>