Show Me the Evidence!
Clinical trials provide evidence for treatments and vaccines for COVID-19 — but what is evidence and why are clinical trials needed?
I am sure that most of you have come across a website like this championing a “natural” way to prevent or treat a disease. But just how much trust can you put in these articles? What is the evidence that the treatment they’re touting does any good, or worse still, is harmful?
There are many different types of evidence used by ordinary people to make those types of decisions. But what exactly is evidence, and how do we know whether we can trust it?
Evidence
The Oxford Dictionary definition says that it is “The available body of facts or information indicating whether a belief or proposition is true or valid”. That definition is complicated but only really helps us if we know what “true” means.
Okay — back to the Oxford Dictionary. True is defined as “In accordance with fact or reality”. So the evidence is the facts, and the truth is agreeing with the facts.
Clearly, this requires everyone to agree on what the facts are, and to interpret them in the same way. However, in a court of law, despite being exposed to the same facts, the jury often fails to agree on a verdict. Luckily, we do not usually require certainty of the guilt of the defendant, only that the jury believes beyond a reasonable doubt that the defendant is guilty.
Most people will have some idea of what evidence means in a criminal investigation, for example, fingerprints, blood spatters, eyewitness statements. But what does evidence mean when we are talking about treatments for diseases?
In fact, health professionals are expected these days to use evidence-based practice. In other words to use the best research evidence available in their practice. Unfortunately, this doesn’t always happen, and there are many other non-scientific forms of evidence still used in health practice today. Here are some of them.
Tradition
I am sure that most of you will have been told by your manager, coworker, or person looking after you “Well, we have always done it this way!” Unfortunately, that doesn’t necessarily make it correct or provide the best evidence for practice.
In 1633, the Inquisition of the Roman Catholic Church forced Galileo, one of the founders of modern science, to recant his theory that the Earth moves around the Sun. The Roman Catholic Church held on to this tradition for 359 years, and it wasn’t until 1992 that the Vatican finally admitted that they were wrong!
Expert opinion
This is me — a Professor. I am an expert in epidemiology. We study infectious diseases. Does this mean you should believe everything I tell you about COVID-19? I certainly hope not! My expert opinion is just that — an opinion. I may be right, but I might also be wrong. In fact, experts often disagree with each other — economists are renowned for this! We also see this in criminal trials where expert witnesses are brought in for both sides and argue against each other. So, no — expert opinion is not necessarily good evidence.
Intuition or belief
What about intuition or belief? You just know this treatment will work!
I once undertook a study of problem gamblers who played the pokies (slot machines). We interviewed many of them, and there was a strong belief that the technique you used to pull the handle, increased your chance of winning!
Logic
Now for one of the favorites, logic.
“I tried this treatment on the patient and he got better, so, the new treatment must be effective”.
Here is a little experiment you could try.
- Put a frog on your hand, and say “JUMP”. Usually, the frog immediately jumps out of your hand.
- Cut off the frog’s legs, put it back on your hand and say “JUMP”. The frog doesn’t jump off your hand.
3. We conclude that cutting off a frog’s legs makes it deaf!
No — logic doesn’t always work either.
And now to clinical trials
So, as you can see, there are many different types of evidence out there, and health professionals still use all of the above. However, the only real acceptable evidence when we’re talking about health and people’s lives, is that coming from clinical trials.
Clinical trials are designed to provide evidence as to whether or not a treatment works. We saw earlier that evidence is closely intertwined with the truth. However, in clinical trial design, we don’t talk about the truth, instead, we use the word bias. You can think of bias as a systematic error (in other words, not a random one), that gives an incorrect answer as to whether a treatment is effective. So well designed trials have a very low bias, and poorly designed trials have high levels of bias.
We often show how good the evidence is for each type of study design by using a pyramid.
Filtered means that someone has reviewed existing studies in a very careful way. Unfiltered means that these are results from original clinical trials.
You can find some of the “filtered” studies by looking in the Cochrane library. I did a quick search for COVID-19, and found five reviews. However, since each Cochrane review usually takes at least a year to complete, it’s not surprising that there are so few.
There other types of systematic review than Cochrane, although Cochrane reviews are considered the highest level of evidence. These are usually registered on the PROSPERO website. At the moment, researchers are working on nearly 900 systematic reviews related to COVID-19.
Finally, if you want to find the results of individual studies (the “unfiltered information”, then the best place to look is the US National Library of Medicine website called PubMed).
Hopefully, you now have a better idea of what evidence is, and what to look for when you’re reading about treatments for COVID-19.
If you have enjoyed this article, please give me a clap, and recommend my articles to your friends! You can read all of my other ones here.
