Peer review: a black sheep in the scientific world?

Peer review is one of the core processes of all of science. With its help grants are allocated, scientists are promoted, research works are published. Together with other issues modern science faces, peer review became a process, whose problems are easier to identify than its advantages.

But first, let’s see what peer review is and how it works in science.

Peer review is not a new notion in science, it has already existed for almost 300 years. It is the keystone of science and is also known as refereeing. The process of peer review resembles a quality-control system and requires all new scientific ideas, discoveries and research projects to be thoroughly studied and critiqued by expert scientists before they become widely accepted. The main responsibility of reviewers is to provide the comments on the quality, originality and significance of the research. The reviewers, of course, are not the highest authority who decides whether a research should be published or funded, but their comments give an important information to the decision makers.

So, how does the process of peer review function? Let us look at the sheme describing the process of peer review.

There also exist several types of peer reviews, that varies according to the known or concealed identities of the reviewers.

• A single-blind review. Reviewers know the author’s identity, but not vice versa. Hiding the identity of reviewers allows them to give comments freely and not worry about dissatisfied authors who want revenge for negative feedback.
• A double-blind review. The identities of the author and reviewers are both hidden. It gives the reviewers an opportunity to focus on the work itself without the influence of any biased attitude to the author.
• An open peer review. The author’s and reviewers’ identities are known to each other. Thus, the reviewers, whose identity is not hidden, are obliged to give more meaningful comments. A big responsibility lies then on a team of reviewers; when they validate and give a green light to an article, it means the scientific subject matter focused in the paper is actual and trustworthy.

Advantages of peer review

Every stakeholder involved in the process of peer review benefits from the approval of scientific results.

For authors, peer review gives the reputation and respect to their work, published in most prestigious journals. Thus it helps the scientist to have chances to be considered more favourably by funding bodies.

For journal editors, peer review influences their decision-making process. An editor can confidently publish an article when it has been closely studied by a team of qualified reviewers. The reputation of the journal is directly connected to the the editor’s management of the peer-review process. The more high quality papers are published, the more reputation and respect the journal will receive.

Being a reviewer is a credit to the scientist. Peer review gives reviewers recognition from different journals that a scientist has expertise in a specific scientific field. Professional and timely reviewing gives opportunities for national or international recognition. Also, it provides a chance to know the latest research that is being carried out in one’s own field by other researchers around the globe. A good and consistent reviewer is not only sought after by good journals but may be offered the editor or editor-in-chief position which certainly is a credit to the scientist.

For other scientists, peer review is a way that helps them to prioritize what they read. There are almost 21,000 scholarly peer-reviewed journals in the scientific world. Thus, by focusing only on the several top journals in their field, the specialist is sure to get the latest and the most substantial papers of the highest quality.

The peer review process prevents a substandard and poor science from reaching publication. Furthermore, the reviewers are basically experts in their field, who are aware of the latest research works. They can, therefore, decline duplicate research and plagiarised papers.

Peer review also generates value for grant applications and University standard textbooks. This helps to ensure that money is allocated only towards feasible research proposals. The peer review of textbooks ensures that students are taught correctly and are provided with trustworthy and up-to-date information.

Another side of the Moon

However, nowadays, there are many scientists who question the value and effectiveness of the actual peer review process. According to them, the negative aspects of peer review far outweigh its benefits. A growing number of scientists are drawing the public attention to evident defects of peer review.

Some experimental studies have been conducted by British Medical Journal (BMJ)[1], one of the most respected peer-reviewed journals in medicine. BMJ editor Fiona Godlee [2] and her colleagues decided to make an experiment: they put eight intentional errors into the paper that was about to be published. Then, the paper was sent to 420 reviewers (221 reviewers responded). The results were quite surprising. The average number of errors identified by the 221 reviewers was two. No one found more than 5 errors and what is more, 16 percent didn’t even find any errors at all!

This suggests that peer review doesn’t really ensure the quality of published research, or if so, only to a small degree.

Along with inaccurate reviews, it is also considered that peer review provokes a lot of biases from the reviewers or editors. The works published in an issue of the Journal of the American Medical Association [3] related to peer review showed the cases of nationality bias, language bias, specialty bias, and even gender bias, as well as the well-known bias toward the publication of positive and negative results.

Recently, the hashtag #SixWordPeerReview became quite popular among the members of the scientific community. There, the researchers share unprofessional and rude feedback from the reviewers:

• Looks ok. Sent from my iPhone. #SixWordPeerReview
• A truly awesome paper, for 1985. #SixWordPeerReview
• You disagree with me, so no. #SixWordPeerReview
• Wow. Much Research. Very Original. Concern. #SixWordPeerReview
• No paragraph explaining paper structure. Reject. #SixWordPeerReview ;-)

Some peer review opponents suggest that peer review also causes the suppression of some scientists’ results. A reviewer who is examining the paper might reject research that conflicts with his own findings or points of view.

In addition to that, there’s the issue of detecting fraud. Generally the reviewers don’t have an access to the actual data on which the research is based. So the reviewers are much more likely to find and identify plagiarism than falsified data.

Quality is not the only issue. The advance of scientific knowledge and progress is significantly slowed down by the process of peer review. Moving the article through peer review system and then being published may take over a year. That’s a long time, especially if the research contains valuable information about a disease or other issue that may influence public health and safety.

In addition to that, the benefits of peer review to the reviewers themselves don’t really outweigh its disadvantages. The reviewers are not paid for providing the reviews and they have no direct incentive to work for hours for free. Furthermore, their names do not appear in the published article.The time spent reviewing could be spent doing something more productive — like original research, for example.

Considering all these problems, the most important question about peer review is not whether to suppress it, but how to improve it.

So, how?

Many initiatives have recently appeared in the response to this issue. For example, during one of the scientific conferences 2016 SpotOn dedicated to “What Might Peer Review Look Like in 2030 a number of recommendations on the improvement of the peer review system was suggested and an increasing number have been tested experimentally.

The propositions include:

• opening up the process;
• blinding reviewers to the identity of authors;
• training reviewers, being more rigorous in selecting and deselecting reviewers;
• using electronic review;
• rewarding reviewers;
• providing detailed feedback to reviewers.

Many of these initiatives are focussed around ‘open peer review’ and function only when the whole scholarly communication system is changing.

Numerous services and innovative platforms that recently emerged to improve the actual system of peer review should also be mentioned in these initiatives. These innovations are used by startups and new open publishers that are applying Web-based technologies like GitHub and Hypothesis. Scirev.Org offers researchers the possibility to share their experience with the review process and point out journals with a speedy review procedure. In addition to that some of the journals use blockchain technology to manage an objective peer review process.

So does DEIP, decentralized research platform. It encourages researchers to provide feedback on their colleagues works in a simple and transparent way.

ET — Expertise Token

Moreover, a researcher, whose work is under review, decides the amount of income from his research, he wants to share with the reviewers. The percentage of this income may reach up to 50% and is decided by the researcher. Thus, the reviewers get extra incentives that motivate them to do a review

Secondly, the process of peer-review is promoted by the protocol to be more time-efficient. When a research is published on the platform, its rewarding window — the period when the contributors to it can be incentivized — lasts only a limited amount of time. Only during this period the reviewers can receive for their review a set amount of DEIP and expertise tokens, allocated by the system.

ECI — Expertise Contribution Index

Finally, all the research works published on the platform are in free public access. It gives reviewers an opportunity to verify the credibility of a particular work. In addition to that, DEIP uses an open peer-review system, when the identity of reviewers and authors are not hidden. The reviewers may ask for any kind of details or proves of investigated data. As a result, the problem of fraud or plagiarism.

Many steps have been taken in the past years to find the solution on the improvement of the peer review process. It makes clear that peer review is an essential part of science and can not be abandoned or left without changes. With its active participation, DEIP will bring its contribution into positive changes of the existing peer review system.

Artyom Ruseckiy, Co-founder, DEIP
Marie Mulyarchik, Editor, DEIP

Follow us

Website | Blog | Twitter | Facebook | Linkedin | Telegram| Forum

Have any questions? info@deip.world

Previous articles:
Blockchain for Knowledge Creation: Green Light for Great Minds
High-quality publication without paywall? Solution is here
Grants applications writing: time-consuming but not always rewarding
funding
Financing Science: a Hot Button for Discussions

Soon we will go deeper into the technical side of DEIP — stay in touch.

1. Schroter S, Black N, Evans S, et al. Effects of training on the quality of peer review: A randomised controlled trial. BMJ. 2004;328:657–8. [PMC free article] [PubMed]
2. Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomised controlled trial. JAMA. 1998;280:237–40. [PubMed]
3. Lee C.J., Sugimoto C.R., Zhang G. & Cronin B. (2013) Bias in peer review. Journal of the American Society for Information Science and Technology 64(1), 2–17.