COMMENTARY

 

Peer review vs Publish all - Navigating the changing landscape of scientific publication

 

 

Mariette Pretorius

Science Communication Officer. GENUS: DSI-NRF Centre of Excellence in Palaeosciences, University of the Witwatersrand, Johannesburg. South Africa

Correspondence

 

 

---

ABSTRACT

SIGNIFICANCE:
This article delves into the critical juncture at which scientific publishing finds itself, examining traditional pre-publication peer review, the emerging 'publish all' model, and the role of preprints. As the scientific community grapples with the need for rapid dissemination of research alongside maintaining rigorous quality controls, the article offers a comprehensive analysis of the pros and cons of each model. It serves as an essential read for researchers, academics, and policymakers, providing insights into how these evolving models impact the quality, credibility, and accessibility of scientific research. It aims to guide informed choices in the ever-changing landscape of scientific publishing.

Keywords: open science, peer review, preprints, publish all, scientific publishing

---

 

 

Being a scientist means conducting research and publishing that research in an accredited journal. At the heart of this practice is the scientific method - a rigorous process designed to produce research that is both repeatable and credible. When your work is published, it is essential that other researchers can replicate your study based on the methodology that you have described. This ensures that both the scientific community and the public can trust your findings, which have undergone a stringent peer-review process in which seasoned experts have evaluated the merits of your research.

Although it is time-consuming and sometimes frustrating, the peer-review process is a crucial gatekeeper. It ensures that your research is not published until it has reached its highest possible quality, making it both reliable and worthy of citation. However, a recent controversial paper about the deliberate burial of the dead by Homo naledi¹ published in eLife, an open-access, post-publication peer-review journal, sparked widespread scientific debate² about current publishing models. Neither accepted nor rejected, the headline-grabbing research now occupies a 'grey' zone created by the collision of sensationalised science with changing publishing and peer-review models. So which is superior: traditional peer review or the emerging 'publish all' model? Should scientists continue to rely on the time-tested but perhaps archaic peer-review system, or should they adopt the quicker, all-inclusive approach to publishing? What implications do these choices have for the quality of published science, and how do we cite future research? In this Commentary, I explore the evolving landscape of scientific publishing by critically examining traditional pre-publication peer review, the emerging 'publish all' model, and the role of preprints, aiming to assess their impact on the quality, credibility, and accessibility of scientific research.

 

Traditional peer review: Review, then publish

The traditional scientific publication process is a rigorously structured pathway³ designed to validate and share research (Figure 1). Researchers submit their completed studies in the form of a written manuscript to a scientific journal of their choice. Here, an editor initially assesses if it fits into the scope of the journal and meets format and language standards. If it passes this first stage, it goes through several more rounds of internal review. It is then finally forwarded to independent experts (usually two or three) in the field for peer review. This framework is known as pre-publication peer review⁴ and has been around for centuries. There are two main categories of peer review - open and closed, the latter of which is more common and further divided into two subcategories: singleand double-anonymous.⁵ In a single-anonymous review model, the identities of the reviewers are hidden from the author, but the reviewers are aware of the authors' identities.⁵ This is the most common method used in many academic and scientific journals but is subject to various disadvantages (Table 1).

The reviewers scrutinise the research for its validity, significance, quality, and ease of reading and understanding, offering (mostly) constructive feedback and recommendations for acceptance, revision, or rejection. Authors may then revise their work based on this feedback and resubmit the manuscript, after which the editor makes the final decision on its suitability for publication. This framework is known as pre-publication peer review.⁶ Ultimately, it ensures the credibility and quality of scientific literature, acting as a gatekeeper against the dissemination of flawed or untrustworthy findings. Research that has been peer reviewed before publication carries a stamp of approval from experts, lending it credibility and trustworthiness.⁶ This is crucial in fields like science, medicine and engineering, where the stakes are high, and unreliable information could have serious consequences. This process also holds authors accountable for their research and conclusions. Knowing that their work will be rigorously examined encourages researchers to adhere to high standards of scholarship, from the design and execution of their studies to the reporting of results.³

While the peer review process is a cornerstone of academic integrity, it comes with its own set of challenges⁷ that cannot be overlooked. One of the most significant drawbacks is the time-consuming nature of the process. The timeline from submission to publication can stretch from weeks to years, posing a problem for time-sensitive research and career advancement. This delay is further exacerbated by the potential for bias within the review process. Personal or ideological biases⁵ can influence a reviewer's judgement, sometimes favouring established researchers, prestigious institutions, or even particular races or genders. Authors are also less likely to report negative results because reviewers are not receptive to them.⁶ Additionally, the limited accessibility of many peer-reviewed articles, often locked behind costly journal paywalls, restricts the dissemination of valuable knowledge. This creates inequities in who can access and participate in the sharing of scientific information and is particularly detrimental for young researchers from emerging countries like South Africa.⁸ The process is also subject to inconsistency and subjectivity, as different reviewers can offer divergent opinions on the same paper. Overall, while peer review has its merits, these limitations highlight the need for ongoing scrutiny and potential reform of the system.

 

Post-publication peer review: Publish first, ask questions later

In contrast to the traditional model, a novel form of scientific publication arrived on the scene in October 2022⁹, when the life sciences journal eLife changed its editorial practice¹⁰, opting to publish every paper sent out for peer review. This post-publication peer review method allows research to be published quickly and then critiqued publicly. Reviewers' reports are published alongside the research, providing valuable context, and allowing readers to make their own judgements about the work's quality.¹¹

This openness comes with several advantages⁹: enhancing transparency, enlightening those interested in the scientific process, and democratising the dissemination of research by removing some barriers faced by early-career researchers or those from less prestigious institutions.

Additionally, authors retain more agency in the publication process, as they can choose whether to implement reviewers' suggestions.¹¹

Meanwhile, preprints have been around for at least 30 years and probably paved the way for the rise of the 'publish all' model. In this approach, researchers upload their manuscripts to preprint servers (like ResearchGate or ScienceOpen) before they undergo traditional peer review. These platforms make the research publicly available almost immediately, allowing other scientists to comment, critique, and even build upon the work. This gives the authors time to incorporate feedback before the paper is submitted to a journal for official peer review. Think of a preprint as a draft version of the research that is publicly shared before it has been officially reviewed.

One of the key advantages of preprints is their speed.¹² Researchers can upload their findings immediately, thereby accelerating the pace of scientific discovery. This speed is particularly crucial for rapidly disseminating crucial health information, as witnessed during the COVID-19 pandemic.¹³ This model fosters a more inclusive scientific dialogue, as more people can access, read, and comment on the research and it particularly benefits early-career researchers¹⁴, who rely heavily on the timely publication of their work to gain recognition for their efforts. Furthermore, preprints are often assigned a Digital Object Identifier (DOI), making them citable, should researchers wish to do so.

However, the 'publish all' model is not without its drawbacks, the most glaring of which is the potential compromise on research quality.¹¹ Without the traditional gatekeeping role of peer review, there is a heightened risk of publishing flawed or even misleading research, which could have far-reaching implications. Even fundamentally flawed, potentially harmful, or unethical science that faces heavy criticism from reviewers¹⁵ will now forever exist online as a citable resource. This absence of a quality filter can affect the credibility of research, as papers published under this model may not be viewed with the same level of trust as those that have undergone rigorous peer review. For this reason, many top-tier journals do not allow authors to cite preprints. Additionally, these publication models place an increased burden on readers to assess the quality of research, rather than relying on the scientific community's judgement. Consequently, researchers may be hesitant to submit their articles to journals that follow this method.

 

The future of citations?

In academic and scientific writing, citations serve a purpose: they give credit where it is due and offer readers a pathway to find the original source for further reading or verification. Crucially, they form the basis of tried and trusted evidence to support new research. Considering the emerging 'publish all' method, how can researchers cite studies presented in this way? Citing papers that have not undergone traditional peer review presents a unique set of challenges and considerations.

In the traditional peer-review model, citing is straightforward because the research has undergone rigorous scrutiny and has been published in a reputable journal. Researchers can cite these papers with confidence, knowing that the work has been vetted by experts in the field. In the publish-all model, like the one adopted by eLife, papers are published alongside reviewer comments, but they may not necessarily have been revised based on those comments. Researchers who cite such articles may inadvertently propagate unverified or flawed findings, thereby affecting the quality of their work and potentially leading to the spread of misinformation. Researchers should exercise caution and critically evaluate the paper's quality and relevance to their work. The citation should ideally include a note indicating that the paper has not undergone traditional peer review, like is done for preprints. Various influential journals (Nature, Science, The Lancet, and the BMJ) now explicitly state that although they allow the citation of preprints, there are specific guidelines about how these materials should be cited.¹⁶ Perhaps, with the rise of the publish-all model, citations should follow the same style as that for preprints, but researchers should be aware that this publication model is more open to opportunistic abuse and perhaps more prone to give credence to bad science.

 

Conclusion

As the academic publishing environment evolves, there is room to revamp and restructure peer-review systems and frameworks. The rise of preprints, particularly during the COVID-19 crisis, demonstrates the advantages of disseminating scientific findings quickly before undergoing conventional peer review. Additionally, the era of open science demands innovative platforms that can facilitate dynamic and thorough debates, contributing to scientific advancement while also promoting fairness and transparency. However, the publication of flawed science using the publish-all model raises serious and valid concerns within the scientific community.

The long-term sustainability of the current peer-review system is increasingly coming into question, even though it is unclear what will take its place, how it will transform or even who will manage this transformation. The era of open science is pushing us toward more transparent, inclusive, and rapid forms of communication. However, this should not come at the expense of quality and rigour. As we move forward, it is crucial to develop frameworks that facilitate quicker publication whilst still being able to uphold the integrity and credibility of scientific research.

 

Acknowledgements

The support of the GENUS DSI-NRF Centre of Excellence in Palaeosciences at the University of the Witwatersrand is gratefully acknowledged. I thank Dr Christine Steininger for sharing her insights and useful suggestions regarding the article.

 

Competing interests

I have no competing interests to declare.

 

References

1. Berger LR, Makhubela T, Molopyane K, Kruger A, Randolph-Quinney P Elliott M, et al. Evidence for deliberate burial of the dead by Homo naledi [reviewed preprint]. eLife. 2023;12, RP89106. https://doi.org/10.7554/eLife.89106.1        [ Links ]

2. Callaway E. Criticism of controversial ancient-human claims tests eLife's peer-review model. Nature. 2023;620:13-14. https://doi.org/10.1038/d41586-023-02415-w        [ Links ]

3. Ali PA, Watson R. Peer review and the publication process. Nurs Open. 2016;3(4):193-202. https://doi.org/10.1002/nop2.51        [ Links ]

4. Elsevier. Publishing guidelines: Peer review [webpage on the Internet]. c2014 [cited 2023 Oct 04]. Available from: http://www.elsevier.com/about/publishing-guidelines/peer-review        [ Links ]

5. Santos A, Morris DS, Rattan R, Zakrison T. Double-blinded manuscript review: Avoiding peer review bias. J Trauma Acute Care Surg. 2021;91(1):e39-e42. https://doi.org/10.1097/TA.0000000000003260        [ Links ]

6. Teixeira da Silva JA, Dobránszki J. Problems with traditional science publishing and finding a wider niche for post-publication peer review. Account Res. 2015;22(1):22-40. https://doi.org/10.1080/08989621.2014.899909        [ Links ]

7. Tennant JP Ross-Hellauer T. The limitations to our understanding of peer review. Res Integr Peer Rev. 2020;5(1):6. https://doi.org/10.1186/s41073-020-00092-1        [ Links ]

8. Pretorius M. Breaking barriers or breaking the bank? The academic publishing system is extorting emerging researchers. Daily Maverick. 2023 September 04 [cited 2023 Oct 04]. Available from: https://www.dailymaverick.co.za/opinionista/2023-09-04-breaking-barriers-or-breaking-the-bank-the-academic-publishing-system-is-extorting-emerging-researchers/        [ Links ]

9. Kelly J, Sadeghieh T, Adeli K. Peer review in scientific publications: Benefits, critiques, & a survival guide. Ejifcc. 2014;25(3):227.         [ Links ]

10. Abbott A. Strife at eLife: Inside a journal's quest to upend science publishing. Nature. 2023;615:780-781. https://doi.org/10.1038/d41586-023-00831-6        [ Links ]

11. O'Sullivan L, Ma L, Doran P An overview of post-publication peer review. Sch Assess Rep. 2021;3(1):1-11. https://doi.org/10.29024/sar.26        [ Links ]

12. Fraser N, Brierley L, Dey G, Polka JK, Pálfy M, Nanni F, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLoS Biol. 2021;19(4), e3000959. https://doi.org/10.1371/journal.pbio.3000959        [ Links ]

13. Guterman EL, Braunstein LZ. Preprints during the COVID-19 pandemic: Public health emergencies and medical literature. J Hosp Med. 2020;15(10):634. https://doi.org/10.12788/jhm.3491        [ Links ]

14. Sarabipour S, Debat HJ, Emmott E, Burgess SJ, Schwessinger B, Hensel Z. On the value of preprints: An early career researcher perspective. PLoS Biol. 2019;17(2), e3000151. https://doi.org/10.1371/journal.pbio.3000151        [ Links ]

15. Callaway E. Sharp criticism of controversial ancient-human claims tests eLife's revamped peer-review model. Nature. 2023;620:13-14. https://doi.org/10.1038/d41586-023-02415-w        [ Links ]

16. Berg JM, Bhalla N, Bourne PE, Chalfie M, Drubin DG, Fraser JS, et al. Preprints for the life sciences. Science. 2016;352(6288):899-901. https://doi.org/10.1126/science.aaf9133        [ Links ]

 

 

Correspondence:
Mariette Pretorius
Email: mariette.vdw93@gmail.com

 

 

Funding: GENUS: DSI-NRF Centre of Excellence in Palaeosciences
Published: 28 May 2024