Shrinking the elephants: moving Open Research practices from recognition to career credit

Photo by Wolfgang Hasselmann on Unsplash

In the first article we wrote (Credit, a troubled elephant in the room of Open Research) we looked at the choices and trade offs facing Early Career Researchers (ECRs) in adopting open practices. This was based on analysis done around the benefits and challenges for ECRs when adopting new open practices in the article by Allen and Mehler, Open science challenges, benefits and tips in early career and beyond. To recap the challenges were the higher standards of rigour, the extra time needed and the unclear incentives. The benefits were the increase in research integrity, the fact that there was new tooling designed to help and the building of future skills. We also highlighted the ‘elephant in the room’ being getting credit for putting the time in to do things properly and the downside of taking extra time and potential clashes with the local academic culture the ECR works in. Recognition is certainly possible for the work done as there is a section of the research community which is onboard with open practices and would like to hear about the use of such practices or even gain more knowledge about certain practices from those who employ them. However the problem of credit still remains — actual career credit — i.e. can using open practices aid in an individual’s promotion, recruitment or attaining other awards; credit is still the elephant in the room.

What is Open Research / Open Science and why is it important?

When we say Open Science we are using the FOSTER definition which defines Open Science as:

the practice of science in such a way that others can collaborate and contribute, where research data, lab notes and other research processes are freely available, under terms that enable reuse, redistribution and reproduction of the research and its underlying data and methods

In my role as Community Lead of the UK Software Sustainability Institute (SSI), I have built and now have oversight of a cohort of over 160 domain ambassadors (the SSI Fellows) who are bringing better computational research practice to their areas of work. Open Research practices such as better code reproducibility and citation have been a key part of what we support the Fellows to do. Getting real career credit for this work is of particular interest to the SSI, as promoting practices that are crucial to research but that people will never be rewarded for doing is untenable and unfair to expect in the long term.

Allowing more transparent research that can be more easily checked for integrity and allow others to build upon seems to the uninitiated the way things might be happening at the moment but it’s actually the area which Open Science aims to address. See Research Culture is Broken; Open Science can Fix It by Rachael Ainsworth (time: 9:30 to 10:05). This confusion is probably compounded by the popular knowledge of the quote, “If I have seen further it is by standing on the shoulders of Giants”. This was a comment by Isaac Newton in a letter to his contemporary and rival Robert Hooke in 1676. The ability to further progress being based on the work of others; the natural assumption is that the work being produced in research is build upon-able in a robust, efficient reproducible way by providing all the materials needed. This turns out not to be the case and it’s this problem that Open Science aims to fix and the area of research culture which needs to have the appropriate credit attached to the extra work needed.

Note, we used the terms Open Research, Open Science and Open Practice interchangeably. Our focus is on research and research practice vs Open Educational Practices used in teaching. The overlap between training offered as part of research practices and teaching offered in University courses is sometimes a blurry line, however the construction of a publication highlighting a hypothesis, current thinking, methodology, results and conclusion is firmly in the space of research.

Who’s responsible for enabling credit for Open Research practice?

How do we shrink this elephant, if indeed it is one elephant and not really a collection of mutually supporting elephants in a parade that supports the status quo. Without being patronising and by way of analogy it’s a point of reflection that making open practices and their adoption, recognition and credit the problems of ECR’s is like asking those starting out in secondary school to be responsible for how the school is run, what the rules are, what’s important and how to get on. They might have ideas and being new starters they will be willing to try things without much prejudice but they are not the ones that set the context.

Returning to Higher Education (HE) it’s the context setters — university management, senior academics, those leading the collection and curation of any research evaluation submission (e.g. for REF) that are responsible for setting the context and rules that will make it easier and more fruitful for ECRs and other researchers to adopt open practices; moving the situation along from recognition to credit.

The problem of Open Practice virtue signalling

The perniciousness of the rush to research results and overemphasizing exciting results should not be underestimated. In the article Beware performative reproducibility,

Stuart Buck (former vice-president of research at Arnold Ventures in Houston, Texas, from 2012 to 2021 and who helped with the creation of the Center for Open Science) highlights the dangerous signal that some adoption of open practices (e.g. pre-registration of analysis to avoid cherry picking results) appear to be performative. They are being done as yet another hoop to jump through rather than with any real reason to follow the philosophy; for example researchers were not following their pre-registered analysis plans and no one was checking. Ultimately he saw it as scientists needing to feel empowered and rewarded for doing robust work e.g. publishing null results and following the data. He saw much idealism in ECR’s and ultimately that it was the role of senior leaders and institutions to create the context where hiring and tenure decisions are influenced by engaging in best practices such as reproducibility and replicability of results; i.e. that the context and career credit for using best and open practices actually exists. A good signal he picked up on was the recent occurrence of university job advertisements now asking applicants about their commitment to Open Science practices.

A similar problem to ‘virtue signalling’ was coined by Michelle Thorne called ‘Openwashing’ — the worries about using ‘Openness’ to appear to be Open but actually not be are not old. The term was created in 2009 due to the city of Berlin in Germany promoting Openness by encouraging people to upload photos of Berlin but it turned out no one could do anything with the collection of photos once uploaded as they were then under closed terms. Kenny Borland speaks in his recent post on Openwashing in HE and how open tools and courses have sometimes been subverted and made closed. By way of analogy, this is a cautionary tale of how Open Practices could be subverted for organisational benefit rather than the greater good of research.

The League of European Research Universities to the rescue

The League of European Research Universities (LERU) is an established network of research intensive universities that develop and disseminate views on research, innovation and higher education through policy papers, statements, meetings and events that help shape policy at the EU level.

LERU published an advice paper in May 2018 titled, Open Science and its role in universities: a roadmap for cultural change. The paper covers 41 different points that cover the eight different pillars of activity as identified by the European Commission’s Open Science Policy (the future of scholarly publishing, FAIR data, the European Open Science Cloud, education and skills, rewards and incentives, next-generation metrics, research integrity, and citizen science). These are to help institutions work out their own strategy, actions and personalised road map to make Open Science a reality in their organisation.

The four overarching high level recommendations to make the change to Open Science are:

1. The appointment of a senior manager to lead Open Science approaches across the eight pillars identified by the European Commission around Open Science.
2. Developing a programme of cultural change to support the change in principle and practices required by the move to Open Science.
3. Establishing advocacy to identify the benefits of the Open Science approach while being realistic about challenges.
4. Development of a communication strategy to increase familiarity with Open Science practices.

Emphasis is mine.

Identifying the elephants stopping credit for Open Practices

Coming back to our elephants; perhaps this is the multiplicity — senior management buy-in, research culture and basic lack of understanding/knowledge. So when we say the elephant in the room of Open Science is credit — what we actually mean is that the elephants in the room are the changes needed to research culture, the increase in understanding and knowledge needed and the senior management buy-in needed to make credit for Open Science a reality. By addressing and ‘shrinking’ these elephants a move beyond recognition towards career credit is made possible making adopting Open Practices a much more incentivised path for ECRs and other researchers.

How are Institutions trying to take forward Open Practice

A number of institutions have started making public statements about how they will take Open Research forward and how they will support Open Research at their institutions. Many are inspired by the LERU recommendations. We highlight some of the statements that came up when we did an internet search for ‘statement on open science’ and at the most popular (according to Google) ones that were put out by HE institutions. Our aim is not to be exhaustive or judgemental but to highlight some of the current statements that are available. Other universities no doubt are developing their plans and will publish and refine them over time; Open Research as the LERU advice paper states involves culture change and that takes time and refinement. There might be newer statements and updates to existing ones at the time you read this article so please do check.

University College London (UCL)

The UCL Statement on the importance of Open Science threads together activities around the Sorbonne Declaration on research data management with a push for data to be open by default. Plan S to move towards 100% Open Access publications for research funded by public grants. And grass roots organisations such as the UK Reproducibility Network (UKRN) focused on research reproducibility and replicability. It also references the DORA principles on research evaluation that calls for the inclusion of all of the facets of the research process including data and software. The statement very much covers recommendations 1, 3 and 4 from the LERU advice paper — covering leadership, advocacy and communication but it does not currently cover 2 — which is the practical steps that the institution will take. One can assume that this area is a work in progress and this is the opening statement in a series.

University of Manchester

The University of Manchester has a Position statement on Open Research, it appears to be driven by the Library; reference to the authors being from the Open Research Strategy Group (ORSG) which is a more central group would have projected a higher level of organisational commitment. The statement encourages people to follow best practices in Open Research by stating ‘While engagement with the principles is voluntary, the University expects researchers to act in accordance with funder mandates’. While these are warming and encouraging they don’t really make any statements about credit but they do say they will support Open Access and other systems with resources and training and the use of the institutional Pure instance. This statement is a positive start on the journey to the Institution being a key supporter of the Open Science approach. The author of this article knows members of the ORSG and they are actively working on policy to address all of the LERU recommendations.

University of Reading

The University of Reading Open Research Action Plan has a focus on developing champions, bids and proposals, a Research Software Engineering pool, specific courses to increase knowledge about open practices and training in software and data carpentry. As well as support there is a clear signal under the theme of ‘embedding open research in institutional norms’ with a commitment to use Open Research criteria in recruitment, reward, promotion and performance assessment criteria. There is also a commitment to nurture and use Open Research culture and practice in their research division, intellectual property policy and staff induction. There is also a commitment to promote Open Research internally to relevant individuals.

It’s encouraging to see that there are some (if limited) resources being attached to these commitments to enable them to happen. The Reading statement is more inline with the roadmap envisaged by the LERU advice paper by addressing all four high level recommendations and is an example of a real move beyond encouraging and recognising good practice in Open Research and building a commitment to give real career credit for the use of Open Research practices. Reading has a solid plan to shrink the elephants in the room of credit for Open Research!

This is an excellent statement by Reading and one that also mentions monitoring to ensure these steps happen; this is a fantastic first step and could well be inspirational for other organisations to use.

Cardiff University

The Cardiff University Open Research Position Statement covers supporting staff who engage in Open Research and a commitment to ‘Recognising the track records of academics in Open Research practices as part of institutional hiring, promotion, and appraisal policies’. While less detailed than the Reading statement on how this will be done — the inclusion of a commitment to give credit for the use of Open Practices rather than just support for their use is a step in the right direction of making it more worthwhile to engage in such practices. It’s interesting to note they reference the position statement of the University of Exeter; this is a great sign that inspiration around how to take things forward is making an impact across organisations.

University of York

The University of York Open research statement outlines their commitment to making research more open by default and mainly covers the motivation and offers support and examples of great practice. The final leap to making a difference for career credit in terms of promotions, recruitment and other awards is not present yet but the structure of the Open Research Strategy Group which provides input directly into the University Research Committee shows high level engagement and commitment that will help develop this area further.

In conclusion

Moving beyond the ECR perspective to the Institutional perspective is important for understanding how Open Research will not only be recognised but rewarded in terms of career credit. The LERU advice paper is an excellent framework to help institutions move forward and institutions are beginning to put out statements and learn from one another’s approach. It’s encouraging to hear about job adverts requiring Open Practice and also the statement of Reading and Cardiff making explicit mention of career credit for Open Practices (recruitment and promotion). The parade of no-credit elephants is likely to shrink with the changes to research culture, increase in knowledge of Open Practices and senior Management buy-in being addressed. A world of more robust, replicable, reproducible and open research is likely to be the winner as well as those who practice these new skills and make this happen. Standing on the shoulder of giants will be facilitated and the now diminished elephants will no longer get in the way.