Science is broken. Being open could fix it.
Part 2 from a 2-part series reflecting on lessons learned from OpenCon 2014.
Read Part 1 here.
In Part 1, I wrote about my disillusionment with science from a systemic perspective. I wrote about how I believed that science was broken. It was broken because it had evolved into an unrecognizable system that was incompatible with what science, in an ideal world, could have been. Its heart wasn’t in the right place anymore. Science seemed to be more about prestige and personal gain, rather than a noble collective effort to expand knowledge.
I had become terribly cynical during my time as a graduate student, but a couple weekends ago, I realized that all was not lost — that science could be fixed. I was lucky enough to attend OpenCon, a conference for students and early career researchers to network, discuss, and learn about how open access and open data could provide novel solutions to the problems we currently face in scientific society. For three days, I was surrounded by pioneers of the open movement, ambitious graduate students and innovative post-docs, all bursting with ideas about how science could be better. Many spoke candidly about how re-thinking science could increase scientific efficiency and quality, increase discovery by embracing collaboration, and most importantly, result in a fundamentally more inclusive approach to knowledge distribution.
These were passionate activists that weren’t just trying to sell a trendy idea. They truly believed in a world where science would benefit the people, rather than a handful of wealthy corporations. They believed in transparency and collaboration, rather than secrecy and competition. And in my humble opinion, I now believe that to become good scientists, we must all believe in, and strive for, these causes. We must all question the status quo. In order to enact change, we must all (in the words of PLoS co-founder Patrick Brown) “be disruptive — and be outlandish!” OpenCon was inspiring and moving, and upon returning, I felt that the least I could do was to share some of these ideas with as many people as possible. So, here I am. In the next few paragraphs, I’d like you to consider how a more open approach to publishing and data could fix various problems in modern day science.
Fixing ACCESS
I have previously written about some of the problematic issues with subscription based models in academic publishing. In short, the majority of publicly funded scientific efforts are hidden behind paywalls and as a result, money is pooled into wealthy publishing companies with humongous profit margins. Compare Elsevier’s roughly 35% annual profit margin to the recent annual profit margins of Apple (35%), or BMW (12%), and you get the idea.
But ultimately, the argument for open access publishing is not just an ethical one, it’s also a pragmatic and more productive one.
Clearly, wealthy corporations should not be the ones benefiting from scientific efforts. Tax-payers should not have to pay twice for an article, and scientists should be able to freely read the work of their colleagues regardless of what subscriptions their libraries have paid for. Research findings should be open access — freely and immediately available online to read and re-use. But ultimately, the argument for open access publishing is not just an ethical one, it’s also a pragmatic and more productive one. As pointed out by Patrick Brown in his OpenCon keynote talk, science is intrinsically non-hierarchical, and the greatest potential scientists might not be at a wealthy institution with a huge library budget, let alone a university at all (see: 17 year old Jack Andraka as a recent example).
By making scientific findings open, we could move towards a more universal, communal, and collaborative scientific community. Every scientist can play a role in shifting the status quo by making their publications freely available to the public by putting them in an institutional repository, or even better, by publishing in a completely open access journal.
Fixing PEER REVIEW
We rarely question the peer review process or its existence, considering it is the only system that most scientists have ever known. Upon closer examination, however, it’s easy to see that pre-publication peer review lacks transparency and can waste hours of redundant efforts. Most dangerously, pre-publication peer review skews publication in a direction that favours fantastical findings and perceived societal importance, over reproducibility and plausibility. (Just think of how many scientists have experienced a paper rejection on the basis of the work not being “interesting” enough for the journal?) This bias is dangerous, because it means that a large proportion of published research findings, could be in fact, false (see: Ioannidis, 2005 for an excellent, in-depth explanation of this argument), and that replications, or failures to replicate (the backbone of science!) are given little acknowledgement or value.
We rarely question the peer review process or its existence, considering it is the only system that most scientists have ever known.
What are some alternatives, then? One clear solution, which has already been adopted by journals such as open access journal PLoS One, is to conduct peer review on the basis of methodological soundness and quality of reporting. This approach highlights the importance of acknowledging null results and replication studies that are often rejected by more “prestigious” closed journals. Another alternative is a more transparent version of pre-publication peer review, where reviewer feedback and reviewer-author interactions are non-anonymously, openly published alongside the scientific paper. This would hold reviewers accountable to their comments, and dissuade them from rejecting a paper on the basis of competition or personal biases. Interesting debate born of the review process could be made available for others to read and consider. Reviewers would receive acknowledgement for their contributions. This model has been adopted by open access journals such as PeerJ, and has been generally well received by the scientific community.
Finally, we should also consider post-publication peer review, where the review process starts or continues after publication — publicly and openly. Here, anyone can read reviewer comments as well as engage in ongoing communal discourse about scientific findings. Examples of this post publication peer review in action include PLoS Labs’ Open Evaluation and The Winnower. Peer Library is another wonderful new project that has a similar mission; in embracing the idea that academic publications should not be “silos of knowledge”, but rather, hubs for continual and active discussion, they provide an online, digital space for researchers and the public to interact with each other, comment on, and annotate scholarly publications.
Fixing TRANSPARENCY
Open data refers to data that is freely, and publicly available to access and to re-use. Many scientists shrink away from this idea, but there is no immediately clear reason as to why researchers should be apprehensive about sharing their dataset alongside their results — unless they have something to hide, of course. Interestingly enough, there is indeed research supporting the idea that willingness to share data is related to the quality of the statistical reporting and strength of results. It seems intuitive then, that enforcing an open data policy could benefit the scientific community as a whole by increasing transparency and accountability, and encouraging scientists to do better research.
There is no immediately clear reason as to why researchers should be apprehensive about sharing their dataset alongside their results — unless they have something to hide, of course.
Making data sets publicly available not only increases accountability, it also presents an avenue for more collaborative and productive research. Perhaps a fellow scientist has a different research question that could be answered by your data set. Perhaps someone wants to re-test one of your hypotheses. The classic publishing system in science merely publishes a shiny static advertisement of your lab’s research output — by publishing your data set — the most significant output of your research, you can share so much more!
There are already a number of platforms freely available for scientists to share their data with others. The Dataverse Network and FigShare are two examples of data sharing platforms that allow use of your dataset to be credited and cited. And we don’t have to stop at just open data. Ideally, the entire scientific process could be more transparent, and more open, from start to finish (and beyond!). For instance, open notebook science refers to the practice of are publicly documenting your lab notes and your entire research process. Making your notes open is an incredibly constructive process for all involved — for example, it allows fellow scientists to pick up on experimental errors you may have made early on in the process, or on the flip-side, allows others to avoid any mistakes you have run into and documented. It also makes your work public more quickly by side-stepping the sluggishness of the current peer review system. Content Mine is a platform that approaches open from the other end of the research process — its overall goal is to extract facts from published bodies of literature through mining. Content Mine can already extract and machine interpret chemical structure diagrams and information from phylogenetic trees in published papers, but one can imagine that the potential further uses offered by content mining are endless!
Science should be universal. Science should be disinterested. Scientists should be asking themselves how they can move collective knowledge — not their careers — forward. And many scientists are already doing just that (I think the number of outstanding platforms and projects I’ve mentioned throughout this post is a testament to that).
The solutions are out there. Science might be broken, but that doesn’t mean we can’t fix it.
Final Note: This piece is a very basic introduction to some of the wonderful things and resources I learned about at OpenCon 2014 — a sort of “Open 101". However, the arguments are more complex and multifaceted than I have space to talk about, and I highly recommend doing more reading and exploring, and watching some of the inspiring webcasts about why we should, and how we can, enact change in the scientific community.
