Open science by design
Report from the National Academies of Science, Engineering and Medicine lays out the promise of open science and the challenges ahead
Earlier this week, the National Academies of Science, Engineering, and Medicine released a joint report into the future of open science. The three academies work together to provide “independent, objective analysis and advice” to the United States government and its people, to “solve complex problems” and “inform public policy decisions”. So it’s a big deal that they’ve come together with open science at the top of their agenda.
Entitled Open Science by Design, the report’s central argument is that open science should be an integral part of the scientific workflow.
Making research results openly available is not an afterthought when the project is over, but, rather, it is an effective way of doing the research itself. That is, in this way of doing science, making research results open is a by-product of the research process, and not a task that needs to be done when the researcher has already turned to the next project.
But at over 200 pages, the report takes some digesting. So I’ve pulled out what we think are some of the key themes and most interesting quotes.
Conducting science openly brings numerous benefits to the scientific and broader community
The report highlights the many benefits of conducting science openly.
Openness increases transparency and reliability, facilitates more effective collaboration, accelerates the pace of discovery, and fosters broader and more equitable access to scientific knowledge and to the research process itself
In particular, open science makes research more reliable and helps address the current “replication crisis”.
Research conducted openly and transparently leads to better science. Claims are more likely to be credible — or found wanting — when they can be reviewed, critiqued, extended, and reproduced by others.
It accelerates progress by allowing other scientists and members of the community to build on research.
When scientific results are made openly available in digital form, they enable faster, deeper, and broader dissemination of the results to other researchers. Wider sharing and collaboration allows research communities to quickly access results and underlying information, which, in turn, stimulates more, and more rapid, scientific discovery.
It also reduces waste, allowing maximum value to be extracted from data.
When data resulting from clinical research on humans and on animals is reused, it maximizes the value of the contributions made by those research subjects to the advance of knowledge.
Finally, open science provides a catalyst for interdisciplinary research, bringing together scientists with complementary expertise, thereby allowing entirely new sorts of questions to be asked and addressed.
Increasingly, addressing complex problems of interest in science and society requires a multitude of methods and scientific results from different communities. This interdisciplinary work will be greatly aided by open, searchable, digital results that are made more available across communities.
Data sharing is a central part of open science but the data needs careful curation
A major focus of the report is the need to improve data sharing.
Discovering, transforming and reusing data collected by others has become a major part of science today, yet the process is still painful.
The Academies endorse the FAIR principles of data management, which state that scientific data should (ideally) be Findable, Accessible, Interoperable, and Reusable. It’s not sufficient for scientists to simply dump their data and expect other people to find it and make sense of it!
There is a misconception in the scientific community — that simply putting experimental datasets in the cloud will make them FAIR. Scientific publications become findable and useable to others only when they are well indexed; scientific data require nothing less.
In other words, data need to be carefully curated as part of the sharing process.
Public access to research data is not sufficient to ensure usability and enable reuse. Uncurated data are often difficult to use. Data curation, management, and stewardship allow for optimal discovery, reuse, and validation of the results of scientific research.
Substantial investment is required in infrastructure for data sharing.
The practices of data curation and dissemination lag behind article publishing in most fields. For data curation and services to be provided routinely, significantly more agency funding will be required. This is absolutely necessary to support the vision of open science.
But this will result in much greater returns on investment in research.
While there are undeniably significant costs associated with implementing policies and practices that support open science, realizing the benefits [of open science] translates into a higher return on the investment of financial and human resources in research activity.
Commitment to open science varies across disciplines
The report notes major advances towards open science in recent years. But there is still a long way to go, with considerable variation in practices across scientific disciplines.
Sharing data, code, and other research products is becoming more common, but is still not routinely done across all disciplines.
This is reflected in the data sharing policies of scientific journals.
For certain types of data in several disciplines (e.g., computational biology, genomics, proteomics), papers cannot be submitted to major journals unless the relevant data have already been deposited in an open domain repository. This has facilitated the discovery and reuse of data as well as the reproducibility of research. At the same time this has only happened in a small number of fields.
In part, these differences reflect the nature of the research being conducted. Projects such as the Large Hadron Collider and the Human Genome Project, for example, have developed a culture of data sharing out of necessity.
Open data is largely the norm in fields such as high-energy physics and astronomy, as funding for these projects is significant, and as such data distribution is well thought out and closely monitored by the respective federal agencies.
The greater challenge is for the “long tail” of smaller projects where there is more heterogeneity in the data types, less regulation, and a greater cost to developing bespoke solutions.
Long tail data exist across all disciplines, mostly only in individual computers or personal websites with minimal or no attached metadata or documentation, resulting in issues such as irreproducibility of research, duplicate research, and, potentially, innovation loss.
Promoting open science will require changes in scientific culture and incentives for scientists
The report notes a number of key impediments to wider adoption of open science by design.
Barriers to more rapid progress include an academic culture and researcher incentives that can work against open science, insufficient infrastructure and training, issues related to data privacy and national security, and the economic structure of the scholarly communications market.
In particular, there is little incentive for researchers to work openly.
The culture of academia does not adequately reward and support researchers engaged in open science practices.
Currently, researchers are evaluated based on the journals in which their work is published (the report views journals themselves as anachronistic).
Journal articles are currently the primary method for summarizing and sharing scientific results, and the journal’s impact factor plays a large role in the assessment of academic achievement. In the digital age, while the journal framework may well continue for branding and content integration purposes, compiling articles in journals for distribution is no longer a requirement for broad distribution.
This means that scientists are disincentivised to spend time or resources producing other forms of scientific output.
Universities and other research institutions should move toward evaluating published data and other research products in addition to published articles as part of the promotion and tenure process. Archived data should be valued, just as the publications that result from them are valued.
Research institutions can take a number of steps to promote open science by design
The report concludes with 5 main recommendations. These include:
(a) changing research culture and rewards;
Research institutions should work to create a culture that actively supports Open Science by Design by better rewarding and supporting researchers engaged in open science practices. Research funders should provide explicit and consistent support for practices and approaches that facilitate this shift in culture and incentives.
(b) providing training for students and researchers;
Research institutions and professional societies should train students and other researchers to implement open science practices effectively and should support the development of educational programs that foster Open Science by Design.
(c) developing policies;
Research funders and research institutions should develop the policies and procedures to identify the data, code, specimens, and other research products that should be preserved for long-term public availability, and they should provide the resources necessary for the long-term preservation and stewardship of those research products.
(d) providing infrastructure for archiving of research;
Funders that support the development of research archives should work to ensure that these are designed and implemented according to the FAIR data principles. Researchers should seek to ensure that their research products are made available according to the FAIR principles and state with specificity any exceptions based on legal and ethical considerations.
and (e) improving collaboration between scientists and other stakeholders.
The research community should work together to realize Open Science by Design to advance science and help science better serve the needs of society.
Scientific culture is a slow-turning ship
Having read and digested the report, I’m fully in agreement with the Academies’ analysis of the challenges facing science and the incredible added value that open practices can bring to science.
It’s important to note, however, that the report is targeted at policy makers, administrators, funding bodies, universities, and other research institutes. As a result, the recommendations are primarily top-down solutions imposed on the scientific community in terms of policies and funding priorities.
Science already operates under various administrative and financial constraints. It’s important that these are realigned in favour of open practices that accelerate progress and provide the widest benefit to society.
But changing scientific culture will take time. The attitude of “publish or perish” will live on long after the implementation of policies and directives that acknowledge other markers of productivity and scientific excellence. And so it’s important to complement these top-down approaches with other more ‘subversive’ strategies.
Open science by default
In a recent blogpost, neuroscientist Björn Brembs argued that the most effective way of promoting open science is simply to provide scientists with tools that have “open” as their default setting.
How do you get scholars to choose open publishing alternatives over legacy publishers? How do you get them to use open evaluation procedures over impact factor? How do you get them to save their data to a repository, rather than on their thumb drive? You provide them, automatically, free of charge and ready to use, with the tools you want them to employ, with the default settings (i.e, open) you prefer.
In this way, scientists are not compelled to share their data, but they have to make a conscious decision not to do so.
Those who have good reasons to make their work closed will balance these against the potential negative consequences (e.g. more time and effort, potential suspicions if everything else is open, etc.) and be able to make their work as closed as it needs to be for them.
Ultimately, of course, we’re all pulling in the same direction. The report from the Academies is important because it provides a context and an environment in which open science is taken seriously. And it provides support to individuals, smaller organisations, companies, and start-ups who are working in different ways to make open science the default mode of conducting research.
Jon Brock is a former cognitive scientist, now science and medical research writer. He is also the co-founder of Frankl, a Sydney-based start-up leveraging new technology to make open science easy and rewarding for scientists. If you’d like to know more, the best place to start is our Medium page.
