Registered reports: A road to reliable research
--
In this essay, Anil Seth, editor of Neuroscience of Consciousness, describes registered reports and other new innovations aimed at improving the credibility of research.
I was in Dublin last week for the biannual meeting of the British Neuroscience Association. Amid the usual buzz of new research findings — and an outstanding public outreach programme — something different was in the air. There is now an unstoppable momentum behind efforts to increase the credibility of research in psychology and neuroscience (and in other areas of science too), and this momentum was fully on show at the BNA. There was a “credibility zone,” nestled among the usual mess of posters and bookstalls, and a keynote lecture from Professor Uta Frith on the three “R’s” and what they mean for neuroscientists: reproducibility, replicability, and reliability of research. The BNA itself has recently received £450K from the Gatsby Foundation to support a new “credibility in neuroscience programme.” Science can only progress when we can trust its findings, and while outright fraud is rare, the implicit demands of the “publish or perish” culture can easily lead to unreliable results, as various replication crises have amply revealed. Measures to counter these dangers are therefore more than welcome — they are necessary.
This is why I’m delighted that Neuroscience of Consciousness, part of the Oxford University Press family of journals, is now accepting Registered Report submissions.
Registered Reports are a form of research article in which the methods and proposed analyses are written up and reviewed before the research is actually conducted.
Typically, and as implemented in our journal, a “stage 1" submission includes a detailed description of the study protocol. If this stage 1 submission is accepted after peer-review, then a stage 2 submission can be submitted, which includes the results and discussion. The key innovation of a Registered Report is that acceptance at stage 1 guarantees publication at stage 2, however the results actually turn out. Also important is that RRs do not exclude exploratory analyses — they only require that such analyses are clearly flagged up. Of course, not all research will be suitable for the registered report format, but we do encourage researchers to use it whenever they can. I’m very pleased that we have a dedicated member of our editorial board, Professor Zoltan Dienes, who will handle Registered Report submissions and who can advise on the process.
Registered Reports are just one among many innovations aimed at improving the credibility of research. Another important development is the emphasis on pre-registration of research designs, so that planned analyses can be unambiguously separated from exploratory analyses. This may be suitable in many cases when a full Registered Report is not. Neuroscience of Consciousness strongly encourages all experimental studies, wherever possible, to be pre-registered. This can be quite easy to do with facilities like Open Science Framework and aspredicted.org. Better science can also be catalysed through publication of methods and resources papers, including datasets. Here again, I’m delighted that Neuroscience of Consciousness has launched a new category of papers — “methods and resources” articles — to encourage this kind of work.
As many have already emphasized, this emerging “open science” research culture is not about calling people out or being holier-than-thou. Like many others, I’ve faced my own challenges in getting to grips with this rapidly evolving landscape. These challenges will no doubt continue, and it’s been uncomfortable contemplating some work I’ve led or been involved with in the past.
Collectively, though, we have a duty to improve our practice and deliver not only more robust results but also more robust methodologies for advancing scientific understanding.
My own laboratory embraced an explicit open science policy several months ago, setting out heuristics for best practice across a number of different research methodologies. This policy came primarily from discussions among the researchers, rather than “top down” from me as the overall lab head, and I’m grateful that it did. One thing that’s become clear is that lab heads and research group leaders would do well to reflect on their expectations of research fellows and graduate students. One well designed pre-registered (ideally registered report) publication is worth n interesting-but-underpowered studies (choose your n). It goes without saying that these changed expectations must also filter through to funding bodies and appointment committees. I am confident that they will.
I prefer to think of these new developments in research practice and methodology as an exciting new opportunity, rather than as a scrambled response to a perceived crisis. And I’m greatly looking forward to seeing the first Registered Report appear in Neuroscience of Consciousness, however the results turn out.
Anil Seth is Professor of Cognitive and Computational Neuroscience at the University of Sussex, where he is also Co-Director of the Sackler Centre for Consciousness Science. He is a Wellcome Trust Engagement Fellow and a Senior Fellow of the Canadian Institute for Advanced Research (Azrieli Programme on Brain, Mind, and Consciousness). He is Editor-in-Chief of Neuroscience of Consciousness(OUP).
