Citizen Science Registered Report Passes Peer-Review
Royal Society Open Science has peer-reviewed and agreed to publish a registered report by Citizen Science Belleville. “Registered report” is a new form of scientific publishing in which the publisher peer-reviews all but the Results and Conclusions sections of a manuscript before the experiment is actually conducted. If the rest of the manuscript passes peer-review, the publisher commits to publish the Results and Conclusions as well, provided the study is conducted as planned. This is reasonable in science because Results and Conclusions should be controlled by the plan.
Registered reports are meant to mitigate bias against publishing negative results, but they also assure the quality of a research plan, and that can be a boon to citizen scientists. Citizen Science Belleville plans to leverage that assurance to crowdfund $10,026, so they can conduct the experiment in Belleville, WI, this fall and winter. Combining registered reports with crowdfunding dramatically reduces risks for funders. No money changes hands unless enough is pledged to cover the entire experiment, and publication is guaranteed when the experiment is complete.
Citizen Science Belleville is a community group open to the general public. It meets twice a month to advance science which promises to improve health, relationships and well-being for future generations. It grew from a bible-study which reasoned that their moral duty to study scripture might extend to study of science, but realized that bible-study groups didn’t exist before scripture was translated into common languages and that science might be as inaccessible today as scriptures were in Latin. The decision by Royal Society Open Science confirms that scientists and science educators have succeeded at making science accessible. It demonstrates that some regular people can understand science well-enough to develop important professional-quality research plans.
Note that the approved research plan is not especially simple. It has the form of a classic double-blind clinical trial with 160 subjects. Additional complications are introduced because it is an independent test of previously published results and because it employs treatments and measures from both endocrinology and political science. One of the original investigators cited the results as indicating ways to manipulate elections, so you’d expect independent testing to be a priority (as least to determine whether elections are being manipulated). The fact that a decade has now passed with no independent testing of such a shocking result is a testament to the complexity of this experiment.
Over a hundred peer-reviewed journals publish registered reports. They cover every scientific discipline (Royal Society Open Science itself is an interdisciplinary journal). Registered reports have not yet replaced the former practice of delaying peer-review until after the experiment is conducted, but it has been suggested as a best practice because it forces scientists to plan more carefully. For independent tests of previously published results, there seems to be no excuse to avoid registered reports.
Citizen Science Belleville has a second way to leverage registered reports: more complete fundraising. If their crowdfunding campaign raises more than the $10,026 they need to conduct the study they have planned, they would like to fund other independent tests of the same result. The original result was obtained in California. After independent testing in Wisconsin, it might make sense to test it in other countries.
All important discoveries should be independently confirmed, so it seems wise for every scientific crowdfunding campaign to raise funds for independent testing (just like raising funds for open access publishing), but additional testing wouldn’t be independent if controlled by the original investigators, so how can original investigators assure potential funders that funds raised for independent testing of their own test will be controlled appropriately? The answer from Citizen Science Belleville is to require that the independent test of their own test be peer-reviewed as a registered report before it can receive funding. This may be another compelling reason to use registered reports.
Please clap for this story and visit https://igg.me/at/cured to learn more about this experiment and the crowdfunding campaign.
