The crises in medical research and what to do about it
There are over 2 million medical research articles indexed on Medline; there are about 30,ooo randomised trials done each year, and annually thousands of new technologies, drugs and devices also make claims of effectiveness. But how much of this research, the trials and the innovations actually translate into real benefits for patients? Not very much.
Twenty years ago Doug Altman, Professor of statistics in medicine at the University of Oxford, published an editorial on the scandal of poor medical research, pointing out that many studies published in medical journals are misleading due to poor methods, unrepresentative patient samples, small sample sizes and incorrect methods. Part of the problem was the plethora of researchers who carry out research — that they are poorly prepared for — to advance their careers. But there were also systemic problems such as misleading reporting of research that needed fixing. Altman pointed out that the system encouraged poor research and what we needed was less research but better research.
Since this editorial there has been a three fold increase in randomised trials indexed on PubMed (over 30,000 trials were published in 2014), and a near four fold increase in observational studies published — 380,000 in 2014. And so we have ended up with more, not less, research. And increasingly much of it is worse research.
A BMJ editorial by ben goldacre and myself pointed out that much of medicine is broken, and that we need to urgently fix the inherent problems in current research: the growth in poor quality evidence is compromising doctor’s ability to provide effective healthcare and preventing patients from making informed choices.
The need for better evidence is highlighted by a number of structural problems that exist with the current production of evidence. Some of these problems are easier to fix than others; but they persist despite numerous attempts to fix them. For example, half of all trials never publish their results, leading to misleading effect sizes as positive trials are likely to be published twice as often as trials with negative results. To fix this AllTrials have called for all past and present clinical trials to be registered and their full methods and summary results reported. Currently, the AllTrials petition has been signed by 89252 people and 703 organisations who think this problem is worth fixing.
Some problems, however, are much harder to fix. Reporting bias, for example, leads to overestimation of treatments effects and underestimation of harms, is a widespread phenomenon but is hard to detect, and often goes unrecognised. So little is done about it.
At the Centre for Evidence-Based Medicine we think part of the solution to the current shortcomings requires pulling together a clear set of achievable goals, and an overview of the strategies that work best. We have started the conversation by creating a manifesto that is developed by people engaged at all points in the research ecosystem who want to engage in fixing the problems inherent in poor medical research.
Our first draft sets out some of the key problems facing medicine today, and the changes we think are necessary. But for every one of the areas we have highlighted in the Manifesto for EBM 2.0, we have five questions for you:
- What change do we want to achieve?
- What actions are currently underway to achieve this change?
- What new actions do you think would achieve this outcome better?
- How would we know we had succeeded?
- Are there any whole categories of problem, or activity, that we have missed?
When it comes to fixing the crises in medical research we have some ideas about what to do; but we think want to gather feedback in as many ways as possible and then publish a final version of the manifesto in The BMJ in time for Evidence Live 2017.