Based on the workshop at the Recomb conference in Paris 2018 and discussions with Epidemium and Recomb participants
Crowdscience and open science programs are crucially important to make research more open, accessible to everyone. Every year, more and more organizations (both private and public) seek help from crowds, try to build new communities to contribute, propose or even fund research projects. A variety of different models exist today.
One of the most popular models are crowd science competitions. Organizations like Xprize, DREAM Challenges, Innocentive organize challenges to solve complex research questions. For instance, DREAM Challenges is a non-profit community that specializes in biology and medicine. Dream challenges excel in building rigorous, reproducible challenges. DREAM leverage on crowds to “find new and better computational models and then make these methods available to all”. At the end of each challenge, the solutions submitted by the crowd are evaluated and winners usually get a reward. The model has a long proven track record since its creation in 2007 and DREAM Challenges have an excellent track of high level publications in Nature, Science attracting top scientists and experts to take part in the competition.
Recently, more open and inclusive open science initiatives gained their momentum. Polymath, Epidemium are just a few examples. These initiatives aim of course to solve the scientific questions but also to build an open & collaborative community. For instance, Epidemium is an open science initiative that aims to understand cancer using open data approach. At the beginning of each challenge, only the minimal rules necessary to start the exploration are set: area of exploration, initial data sets, potential area of interest and scientific and ethical guidelines to respect. The rest is up to participants & to the community in general with of course highly active presence of the Epidemium’s coordination team. What appeared unique in this program is the capacity of the project to bring together a variety of different participants : data scientists, medical doctors, sociologists, graphic designers, researchers, students, etc. Some of them were experts on cancer or data but definitely not all of them. Being more inclusive does not allow to guarantee outcomes especially in terms of relevant scientific results to the community at the end but allows to account for transdisciplinarity and establish bridges between different disciplines and non experts.
In both crowd science competitions and more open & collaborative projects, a community of participants is the key. The community of participants, so called solvers are critical for the overall success of these programs. It is crucial not just to engage them but make sure that they keep participating and feel engaged. For that, we needed to understand why they have participated in the first place? Why some of them abandoned? What was important for them?
Recognizing the importance of respective communities that DREAM Challenge and Epidemium have built, we decided to ask participants directly. The insights were gathered during the RECOMB meeting workshops and during the individual interviews with some of the participants.
As a result, we have learned a lot from the community.
What drives participants to take part of open crowd science projects?
It will not come as a surprise but solvers of both programs stress out learning as a key driver for their participation. This learning is of different nature: learning an entirely new field, new methods to approach solving problems, learning from others and collaborating with other participants, etc. Open science programs provide possibility to deal with interesting and scientifically important topics where participation ranges from a desire to win to simply contributing and being part of something bigger : “it gave me the chance to develop something”, “I could work on unique data sets”, … Some of the participants were motivated by working on the cutting edge of technology & science. Dream benchmarks are recognized for their quality and help participants to extend the state of the art.
Apart from learning and testing new methods; improving their skills; participants learn to work in teams (both inside their project and also during the challenge more generally : community phase during the Dream Challenges or RAMP sessions during the Epidemium programs); to develop their leadership skills.
For young scholars, crowd science challenges offer the opportunity to find jobs, for all the other participants (especially during the numerous conferences and meetups organized by Epidemium) to meet new people, to extend their network. For isntance, in Epidemium a possibility to access to the world of research and know different institutions that conduct research related to cancer was highly appreciated by professionals from other domains.
By providing tools, data, expertise, Epidemium appeared to participants as a great way to test their ideas, to develop projects with serious means and valuable guidance without taking personal and professional risks. The only asset participants have invested was their time.
What are the eventual drawbacks of their participation?
One of the major drawbacks mentioned by solvers is lack of transparency from the organizers and unclear communication. The majority of solvers interviewed stress out the importance of setting up clear rules right from the beginning and don’t change over time (standardized rules, selection criteria, incentives, duration, etc.). For crowd science challenges, participants indicate that topics are somehow unclear for some of the participants and they would like to be able to evaluate the problem complexity and have access to the relevant documentation (related to the topic) right from the start. Duration of the challenges seems to be too short as well. The overall infrastructure to conduct experiments should be reliable.
Organizers should be aware that participants invest their time and energy to these programs additionally to their normal duties. They feel overwhelmed by high amount of information and requirements to follow : “These challenges are quite hard from life/ work time balance : you need to find free time to participate”.
Epidemium participants have highly appreciated the possibility of meeting people from a variety of different disciplines but they also indicate the difficulty of starting the real collaboration and going beyond simple networking. We believe that organizers role is crucial here, they need to establish settings adapted to transdisciplinary projects explorations.
In the final stages of the challenge and post-challenges, some of the solvers might benefit from organizers and other participants’ expertise in finalizing the projects, helping them write and publish their work, etc. In case of crowd science projects if the outcome is the publications, it should be clear from the beginning how the authorship works (first, last author, alphabetic order or not, etc.). Participants mentioned that it could be great to have linear authorships in publications. Moreover, publishing methods and data could be a great option as well. What if special issue challenges existed? Everytime we launch a new open science program, we guarantee that the best results will be considered for the challenge based issue.
In case new teams are formed for the program, there is a loss of momentum at the beginning before the team becomes efficient. Participants would like to have like minded teammates or additional onboarding and coaching sessions at the beginning of the programs.
Having unique data & reliable data is a well known challenge and having an access to larger data sets of often problematic but might be crucial for the program to succeed. It might be interesting to explore multiple partnerships to have a better data access.
Overall, the point of making open & crowd science initiatives more prestigious and legitimate would be valuable for participants. It would give them more arguments to valorize their participation (What if open science Nobel prize exist?) and will motivate them to spend more time on their projects.
We do have a paradox today with dedicated short term challenges where participants invest a lot of time to compete in order to win the prize and more open & inclusive participation where they feel attracted to the cause but participate whenever they have time to do it. The question is : How to balance the two while creating the positive outcomes in the long term?
Participants’ feedback is crucial. We will be happy to hear about your experience in participating or organizing open & crowd science initiative. Please contact us at ckmap@epidemium.cc
Discover and take part in Epidemium: site & plateforme.
