The MetaROR Publish-Review-Curate Model: our experience as authors

Written by Hans de Jonge, Jeroen Sondervan and Bianca Kramer.

Challenges of the traditional publishing model

The conventional publishing system with its traditional peer review model has long been criticized for being slow, opaque, biased and inefficient. Decisions on acceptance/rejection are typically made behind closed doors by a small group of editors based on confidential peer review reports. Lengthy review cycles delay the dissemination of research, while repeated review requests for rejected manuscripts put a strain on an already overburdened peer-review system. Finally, the focus on acceptance/rejection as a binary often stands in the way of a constructive dialogue about the research reported on.

Enter the PRC model

In response to these issues, the Publish-Review-Curate (PRC) model has emerged as a promising alternative. It aims to speed up the sharing of research results, make evaluation processes more transparent and inclusive, and use community resources more effectively. The PRC model breaks the publishing process into three distinct phases. In the Publish phase, articles are shared as preprints. In the Review phase, reviewers evaluate these articles and their reviews are made openly available. In the Curate phase, articles (along with their reviews) are included in curated collections, similar to publication in traditional journals. Importantly, each phase can potentially be handled by different platforms or services. The picture below compares the PRC process with traditional ways of publishing.

Several platforms already adopt variants of this approach, including Copernicus, F1000, eLife, Scipost, Review Commons, Peer Community In (PCI), PREReview, Society and others. MetaROR is a newer initiative focused on open review in metaresearch, launched as collaboration between the Research on Research Institute (RoRI) and AIMOS: the Association for Interdisciplinary Meta-Research and Open Science.

Why we chose MetaROR

Earlier this year, we wrote a paper on the use of open metadata to track publications under so-called Transformative Agreements. This work is an offspin of our work on monitoring the open access policy at the Dutch Research Council (NWO). Since in our jobs we are responsible for shaping policies on (open) scholarly communication reform, we were curious to experience the PRC model first hand.

Our experience with the platform

The first step in the process was the publishing of our article. For that we chose MetaArxic, the preprint server dedicated to metaresearch. We shared the data and code on Zenodo, in line with MetaROR’s requirements. The review phase started by submitting to the MetaROR platform which provides a very straightforward and user-friendly portal. No lengthy uploading of PDFs. Apart from some basic metadata, we only had to provide the DOI to the preprint. MetaROR also delivered on speed. Within 13 weeks, we received two signed reviews from experts and the overall assessment of the handling editor confirming that our paper would be published (“curated”) on the platform along with the reviews. 

We appreciated the transparency of the open, attributed reviews. Not only because it helps to better understand the reviewers’ perspectives and points of critique, but also opens the door for potential follow-up and dialogue with the reviewers, and provides readers with valuable context in reading and assessing our preprint. 

Slightly disappointing was the fact that MetaROR does not support posting a revised version on its platform without going through another round of submission and review. It didn't feel quite right to claim time from already overburdened reviewers with a renewed submission. However, we were offered the option to submit an author response, so the entire correspondence about the review process is available online. We also decided to incorporate the suggestions by the reviewers in a new version of the preprint on MetaArXiv and included a link to that in our author response. 

Recognition and partner journals

Would the publication of preprints together with open reviews on MetaROR be attractive to researchers? Or do they still feel the need for the recognition that comes with ‘traditional’ journals? For these cases, MetaROR has developed the concept of so-called "partner journals": journals that commit to consider papers that have been reviewed on the MetaROR platform for publication in the journal, by reusing the available open reviews. 

We were curious to see if journals would indeed accept already available open reviews, so we decided to put this to the test and took the extra step to submit our article to Quantitative Science Studies (QSS), published by MIT Press — one of MetaROR’s partner journals. We were pleasantly surprised to receive a decision within 6 weeks, based on the open reviews from the MetaROR process. Our paper was accepted for publication with minor revisions, and two editors provided a few very useful extra suggestions for improvement. We understand we are the first authors to have an article accepted in a journal as a result of a partnership between the journal and MetaROR. 

Would journals outside this partner-journals network also have accepted our paper based on the open reviews? We may never know, but our experience is an encouraging example of how this process could  work. 

Reflections

Overall, our experience was very positive. Yes, it was slightly roundabout — we could have gone straight to QSS or just kept it to the posting of our preprint on MetaArXiv. But trying MetaROR gave us valuable insight into a model we see as an exciting step forward for open scholarly communication. For researchers by profession, PRC platforms may still feel too niche or not provide the recognition required for career progression. Even for us, acceptance by a “traditional journal” felt rewarding in a way that MetaROR alone did not.

This made us even more aware of what we knew already: for innovative publishing models like the PRC model to become mainstream, it is imperative that researchers are recognized and rewarded for engaging with them. Reforming scholarly publishing must go hand in hand with reforming research assessment and incentives. Without that, new publishing models will likely remain limited in impact.

What we most appreciate about the MetaROR approach is that the value of peer review, and the transparency of the peer review process, is decoupled from downstream decisions on acceptance/rejection (i.e. curation) by journals. Even if our preprint had not been accepted for publication in QSS, it would have been available for public scrutiny together with the review reports. 

If we have to name one critique, it concerns the use of DOIs. MetaROR currently mints separate DOIs for reviews, editorial assessments, and author responses, but not for the publication of the article itself on the MetaROR platform. Although as authors we understand this would mean that a preprint with the same contents would exist with two different DOIs, an overarching DOI for the full package (preprint, reviews, author response and link to updated preprint) would make the process more coherent.

Conclusion

Our first-hand experience with the PRC model, via MetaROR, showed us both its promises and its current limitations. It demonstrates how scholarly publishing can be faster, fairer, and more transparent — but also how dependent its success is on changes in recognition and rewards in research assessment. For us, the journey was both rewarding and eye-opening.

• Hans de Jonge is director Open Science NL, part of the Dutch Research Council NWO
• Jeroen Sondervan is programme leader for Open Scholarly Communication at Open Science NL and responsible for the open access policies of NWO
• Bianca Kramer is executive director of the Barcelona Declaration on Open Research Information and independent advisor and research analyst at Sesame Open Science