Scientific publishing and reviewing — thoughts from an ecosystem-service perspective
Before the ESP10 conference started, YESS organized a workshop day for early-career researchers (ECRs). In the afternoon, we sat down with Berta Martín-López, Alexander van Oudenhoven, Benjamin Burkhard, and Matthias Schröter who all are editors(-in-chief) of journals that regard ecosystem services and talked with them about scientific publishing and reviewing.
What do you see as the current developments in scientific publishing?
Benjamin: Open access and new possibilities through online development such as collaborative online writing tools and new innovative journals/publishers emerging.
Alexander: Different journals, platforms and institutes are trying out different things to go beyond the traditional way of making impact (i.e. beyond citations and publishing in high impact journals). Think of using (and analysing) social media outreach, blogs, videos, press releases, illustrations and non-scientific writing based on research findings.
At the same time, we are noticing interesting discussions around open access. More and more funders and research institutes require research to be shared open access, which is by itself logical and commendable. Open access will prove to be a success if the quality of published research will remain to be reviewed ethically and transparently, and if researchers from all over the world will have the opportunity to submit and publish their work.
Finally, there is a continued discussion about how to reward reviewers. Ironically, many journals don’t give reviewers enough time to actually review manuscripts, because of their desire to publish quickly. This often results in insufficiently reviewed manuscript, which provides the scientific community a disservice. Although we rarely struggle to find reviewers, thanks to our amazing Editorial Board, we are continuously experimenting with reviewer recognition, and we will be running a pilot soon on this topic.
Berta: There are also some innovative developments associated with the publication formats and with the structural setting of the review process.
Regarding the publication formats, we are witnessing an increasing diversity of possible publication formats. In addition to the more mainstream research papers, short comments and reviews, many journals are promoting original formats to facilitate dialogue within and across research communities.
Concerning the structural setting of the review process, we are witnessing new approaches seeking to counteract implicit biases associated with gender. For example, more journals have implemented double-blind review to favour female authors to get their research published. In addition, few journals (e.g. Ecosystems and People) are fostering gender balance in their editorial board in order to invite more women as reviewers regardless the suggestion by authors. Likewise, Ecosystems and People seeks to represent different regions of the world through the members in the editorial board. These two measures promote more fair reviewing processes since they account for gender and cultural diversity.
Matthias: Open access is at the moment the most important and exciting topic in scientific publishing. This changes the business model of publishers quite drastically. Now the amount of published research influences their turnover, and hence there is an incentive for new business models (speak: predatory journals) to publish a lot. It is our task as scientists to ensure rigorous peer-review to prevent over-publishing. There are some examples in the field, journals with impact factors, which have hundreds of special issues lined up per year. I doubt that this is the right way to go. Neither do I see charging 3000 Euro for publishing one paper open access in a non-open-access journal as a sustainable business model.
Rewarding reviewers, as Alexander said, is an interesting and important field. I know that in some fields reviewers are paid. In principle I am in favour of such rewards, but we need to establish standards. For instance, when I see reviews consisting of a couple of sentences only, is that worth 50 or 100 Euro? Payments could hence create incentives that could even impair the peer-review process, when people accept all reviewer invitations, just for the money.
What are the main advantages and disadvantages of the current scholarly publishing system?
Benjamin: Advantages are peer review as a way of cross-checking each other’s work and knowledge sharing via Open access. Disadvantages are the difficulties to find and motivate reviewers and partly extremely high costs for Open access publishing (certainly with established publishers).
Matthias: I think that we have become too dependent on some single large publishers. The situation for many German scholars at the moment is that our institutions do not have access to Elsevier journals due to stuck negotiations. This has led to a situation in which many of my colleagues decide to not edit, review or write for Elsevier journals. This is understandable and unfortunate at the same time, because sometimes a paper does fit a specific established journal well, and also a manuscript would be very suitable for that one German reviewer. That means that negotiations about costs and profits have the potential to hinder good science.
The journal impact factor metric has several limitations. However, impact factor is quite established and many funding bodies/universities take it into consideration when evaluating researchers. Do you see any good alternatives? What do you think is the way forward to assess the quality of researchers and their publications?
Benjamin: Impact factors are only one indicator to assess the “quality” or “relevance” of a journal, it does not necessarily tell much about each article’s quality. Perhaps it would be better to check citations of each single article instead of doing it for journals and across all articles. Researchers and funding bodies should be motivated to try new publishing format also.
Alexander: If you publish a paper in a high-impact journal (good reputation, high impact factor) but few people read it and even fewer cite it, then how can you claim that you have published a high-impact paper?
Sometimes opting for a journal with less impact but a different audience and scope can increase the impact of your paper. I would recommend researcher to always monitor the impact of their individual paper, and do this in the broadest sense possible; downloads, views, Tweets, shares, Mendeley saves, blogposts, interviews, web articles, policy briefs, policy uptake, and media coverage. This doesn’t happen by itself, so we most also look to make that impact. Also note that most universities, research institutes and some funding bodies are relatively slow when it comes to following trends around impact. So please communicate your impact to them in an original way too, so that they become aware of it.
Ecosystems and People has the philosophy of ‘impact per paper’, and we think along with the authors how to achieve that. You are personally responsible for the impact of your research, but we can help you with achieving that impact. For instance, recently a group of enthusiastic YESS scientists published their paper on ‘Disentangling NCP and Ecosystem Services’. We believed in the potential impact of that manuscript and even granted a waiver for the publication costs, but kept providing critical feedback throughout three review rounds. Although only published a month ago, it already has amassed 2,500 downloads and has reached a potential Twitter audience of 150,000 through 201 tweets (Altmetric score 79). I would call this high impact, and this is entirely due to the enthusiasm of the authors (on different platforms) and the pleasant cooperation between authors and editors.
Matthias: I also do not like impact factors that much, but as early-career scientists we have to play by the rules as long as they are considered important. Paper-based citation analysis is a better indicator, next to alternative metrics such as being picked up by media or policy documents. I think that for assessing the productivity of a researcher at a glance, we need a mix of quantitative indicators, and a quality check: what is the vision; do the papers (together) tell a story and bring the field forward? But I acknowledge that for committees this might be hard to achieve due to time constraints.
Which are the ethical issues that you as editor most frequently have to deal with?
Matthias: This is a very exciting part of the work, as it is very diverse. As a reviewer I have seen a team citing itself in almost 1/3 of all references. As editor I have seen reviewers trying to impose (too much) their own ideas on a paper. Open review is an exciting field, and I think that both double-blind and double-open (after acceptance) peer-review have some advantages. The least preferable option is, I think, when a reviewer can see who the authors are, but not the other way around.
Benjamin: Usually the issues are with reviewers, who do not provide fair and constructive reviews. Sometimes it seems that researchers seek ways to “destroy” a study that they don’t like (for whatever reason) instead of providing help to improve the manuscript.
Alexander: We often find that submitted research can simply not be replicated, which is often due to poorly written and incomplete methods. In addition, methods are often proposed without (self)reflection, without awareness of alternative methods out there and the shortcoming of your own. Also, we are critical when it comes to advancing your own research approach (including self-citation) without reflection.
The same holds true for reviewers, who suggest citations to their own work or pushing their own research agenda by asking authors to reflect on aspects that are out of the scope, or simply criticize work because they obviously don’t like it. This is the foremost reason why we adhere to double-blind peer review; to protect the fair and content-based review process and the authors that propose novel integration of scientific methods.
Berta: One of the main current challenges of the research on human-nature interactions, social-ecological systems and ecosystem services is that there are no clear ethical guidelines. While disciplines with long traditions have developed ethical scientific standards, new research traditions that rely on inter- and transdisciplinary approaches are lacking of such guidelines. It is essential that the scientific community works towards developing these ethical codes. In the meantime, we should adhere to a few ethical principles when conducting social scientific research on ecosystem services and human-nature connections. As editor and author, I personally adhere to the following ethical principles when working with social actors: (1) Principle of Full Disclosure, (2) Principle of Prior Informed Voluntary Consent, (3) Principle of Confidentiality, (4) Principle of Active Participation, (5) Principle of Reciprocity and Mutual Benefit and (6) Principle of Acknowledgement and Due Credit.
How does your journal handle aspects of transparency (including in reviewing, data use, production costs)?
Benjamin: Our review process is community-based. Pensoft journals support the open science approach in the peer review and publication process. Reviewers are encouraged to disclose their identity to the authors and consider supporting the peer review oaths, which tend to be short declarations that reviewers make at the start of their written comments, typically dictating the terms by which they will conduct their reviews.
One Ecosystem also involves subject editors in order to distribute responsibilities for each article differently and across various people, backgrounds and scientific disciplines. Publication fees can be found on the website and are comparably low.
Alexander: Through our publisher [Taylor & Francis], we are a member of the Committee on Publication Ethics (COPE), which pretty much covers all things ethics and transparency. We make use of a double-blind peer review process, because it makes sure that research is being judged based on its intrinsic quality merit, rather than on which authors are behind it. The two most ideal models would be either double-blind or entirely open, and things can be said for both. We share the reviews with all involved reviewers (anonymously), as well as the recommendation and decision that were based on their reviews. In this way, we provide reviewers with reflection on the usefulness of their review, as well as ownership in the decision-making process.
Also, it is important to understand that issues around publication costs etc. are literally none of our business, nor of my colleagues. We are academic editors, and thus responsible for maintaining a clear direction and vision in terms of the journal’s scope and how we handle submitted research. Publication and production costs, and even data use, is the responsibility of our publisher. We have a rough indication of the production costs, and if we didn’t agree with this model we wouldn’t be investing so much time and effort with the publisher. Also, we did choose for the Open Access model with our publisher, however, because the publication costs are at the lowest spectrum, and because ample options exist to (partially) waive the publication costs.
How is the reviewing process working in your journal? Which are the main review criteria?
Benjamin: The whole review process is community-based and transparent, including disclosure and options for direct communication/commenting between authors and reviewers. The review criteria can be found from the website.
Alexander: Together with Matthias Schröter and the Editorial Assistants, Berta and I check the quality of the submission and the match with our journal’s scope. If in order, we send the manuscript to our Handling Editor that has the best match with the paper’s subject. The Handling Editor will then invite reviewers. The invited reviewers will always be a combination of ECRs and more senior researchers, and we aim for gender equality and geographical representation as well. Once the reviews are in, the Handling Editor will formulate a recommendation, based on the reviewers’ verdicts as well as the editor’s own observations. Finally, we formulate a decision based on the reviews and recommendation, thereby critically assessing all arguments and reviewing the manuscript ourselves too. In our decisions, we put emphasis on how and why the manuscript could be improved, thereby relating the paper’s topic to our journal’s scope. This could lead to multiple revision rounds, ultimately until we are happy with it. This does mean that we (re)view the manuscript multiple times (and multiple rounds), which is a considerable time investment.
What expectations do you have from the reviewer?
Benjamin: To be constructive, fair and transparent with the main aim to improve the quality and content of the reviewed article and to support the authors, no matter at what career stage they are. The review process is part of a scientific debate and should foster scientific progress, not the development of individual egos.
Alexander: We expect our reviewers to be independent, familiar with good scientific conduct, and fair. If you show no respect for the research(er), we will not consider your review. Reviewing critically is not about being negative, it is about suggesting strong points and how to build on these merits to make the manuscript worth publishing.
Matthias: I expect fairness, and also being open about which part of a manuscript they could not review properly. I usually would say that a detailed review can have three to five pages. Much shorter reviews tend to miss out important points. I expect to reply to an invitation within a couple of days, and I can live with a delay of the review of a couple of days usually. It is important to communicate about it. The system only works if all of us contribute with fair reviews, as we would like to read them on our own papers.
How can ECRs engage with your journal?
Benjamin: ECRs can submit articles, sign up as a subject editor (via the online form), sign up for the reviewer data base, or edit a special issue/article collection.
Alexander: We were approached during ESP10 by several YESS members with exactly that question, to our delight. First of all, you can register yourself as a reviewer, with our journal as well as other journals. Make sure to specify good keywords so that editors will find you. Furthermore, if you have an online profile, we will be able to find you. So, make yourself discoverable, through your institute’s website (provide keywords!), your profiles on Mendeley, LinkedIn, Scopus, Twitter etc.
Also, you can engage in scientific discussions by writing perspectives. We especially welcome perspectives from young researchers and those from outside Western Europe and North America, or different scientific disciplines than are currently mainstream in ecosystem services research. If you have any ideas for papers that are perhaps slightly unusual, please do get in touch with us!
Also, two ECRs, Sakshi Rana and Patricia Santillán Carvantes, currently help us with managing the review process. They help us and we help them by informing them about the process, the do’s and don’ts and by exposing them to different research and perspectives.
Do you follow a strategy with your journal to engage ECRs?
Benjamin: Not specifically, we really try to engage researchers from all levels, including ECRs.
Matthias: I usually invite at least one ECR for a manuscript. The quality and depth of the reports is usually very high. Moreover, there seems to be a higher chance they accept an invitation.
Alexander: Since May 2018, many early to mid-career researchers joined our Editorial Board. They have brought enthusiasm, familiarity with novel scientific advances and a large network with them, leading to very efficient and high-quality reviews. Also, we aim to have half of the reviews to come from ECRs, as ECRs tend to be on top of the latest developments and are keen to provide in-depth reviews.
Because we particularly welcome submissions from ECRs, we make sure that they receive fair reviews, and prevent any bias based on seniority or other aspects. The double-blind peer review system tends to help in this matter, especially considering the highly interdisciplinary studies that are usually submitted by ECRs.
Finally, although we provide waivers to promising research that is proposed to us, we provide fairness when it comes to waivers. This means that we do not provide waivers because authors are ECR but only because of potential of the work and match with our scope.
By Klara Winkler
McGill University
