Science Kills!
In the last semesterbreak I was proofreading the Bachelor thesis of a friend. He conducted a study about the influences of excessive breath holding on subsequent cognitive activities. A huge part of it concerned ego depletion, a psychological phenomenon based on the idea that willpower is a limited resource which can be depleted by both mental and physical activities and needs time to refill again. When willpower is exhausted we talk about the state of ego depletion. Subsequent demanding activities cannot be executed properly unless willpower recovered again (more on this on Wikipedia).
So this was the first time I heard about this theory and it actually sounded pretty conclusive to me. However, in his empirical study my friend could not find that holding breath is an activity requiring willpower. This was quite sobering for him, as he felt like his study is not meaningful because it could not prove his hypotheses. Anyway, a couple of days after he handed the thesis in, I found an article in Spektrum der Wissenschaft (spectrum of science) also concerned with ego depletion. It states that a new study of 24 independent research teams could not provide any evidence for the psychological phenomenon. This is even more more surprising since one of the founders of ego depletion, Roy Baumeister, was consulted as an advisor (apart from that, in this English article — which gives a more detailed overview of the whole story — Baumeister says he still believes in his discovery and criticises the methods of the new study). However, the crucial point of the article is not whether ego depletion is real or not, but that Psychology and other sciences are in a crises. Another study from 2015 is mentioned which reproduced experiments of earlier studies in order to get the same results — because that is how science is supposed to work. Shockingly, the outcome showed that only less than half of the psychological experiments were repeatable. Of course, both these studies left space for criticism themselves (and they have been criticised for various reasons), but they revealed that there are studies which cannot be reproduced, results that cannot be generalized, and you can still find such studies to be the foundation for scientifically accepted theories. Furthermore it shows how minor discrepancies or mistakes in methodology can lead to completely different and, at times, wrongly interpreted results.
The article that reminded me of this topic (which finally lead to me writing about it) appeared in Die Welt some weeks ago and shows that methodological mistakes and misinterpretations of results can even lead to fatalities, or, more precisely speaking, to not being able to prevent fatalities. Some years ago, a large scale study tried to find out if PSA-tests could indicate prostate cancer early enough for vital treatments. For this reason they compared two groups of about 38,000 men each. The PSA-test was only used in one group (the control group was not treated at all). After about seven years both the PSA group’s and the control group’s mortality rates were compared. The researchers could not find any differences, so PSA-tests were considered useless. Other studies had different results, but this large scale study’s conclusions were believed to be true by many (and obviously many insurance companies)— only until a short time ago two urologists found a mistake in the study. Although they were recommended to not do so, 9 out of 10 people in the control group privately conducted PSA-tests (frankly, who can be mad at them for trying to improve their chance to live a longer life?), so no wonder no differences were found. Now many medic scientists have been reassured of the positive effect of PSA-tests and assume that this mistake has and will cost thousands of peoples lives.
These examples open up how unconscious methodological flaws can impact an entire scientific discipline and even the lives of people, but above all there is a lot of conscious manipulation in scientific research, especially in psychological and medical sciences (at least that is what Die Welt and Die Zeit say). Die Welt quotes another article by the medical journal Lancet which says 80% of all biomedical research are “rubbish” and not reproducible. On the one hand the system is criticised. If scientists want to keep or improve their positions, they are put under pressure to publish new research results. Journals preferably publish articles which reveal something spectacular and groundbreaking. This leads to, on the other hand, researchers manipulating data and sugarcoating results to make their research more attractive to be published. What they do not seem to think about is which consequences these manipulations might have. Maybe they are wrongly creating a new field of research, maybe their results will lead to the death of people. We will probably not know in most of the cases. Anyway, there has been a lot of cases become public in the recent years, and what we definitely know is that researchers not working scientifically are undermining the credibility of science as a whole.
