“The case for doing less in our peer reviews” by Kate Derickson is an interesting essay on scientific reviews.

While it is a luxury to receive thorough and carefully thought-out comments from a colleague, the nature of blind peer review means that the author cannot know who is making suggestions […] And yet, the author is often relying on the paper being published for professional security or advancement. This puts the author in the position of being obligated to rework their arguments according to constructive suggestions made by an anonymous person whose credibility or self-interest they cannot assess. Moreover, while reviewers often identify similar issues in a paper, they often propose a variety of different approaches to addressing them, many of which work at cross purposes. Authors can be overwhelmed by the range of suggestions, feeling obligated to split the difference and cover all the bases in case the paper goes back to all three reviewers. While papers generally get better through the review process, authors often have a difficult time navigating contradictory reviewer suggestions.

But wait, there is also a point where I do not agree (in the light of the recent elife decision).

we think carefully about what we decide to send out for peer review, in order to enable us to curate a table of contents that we think is at the cutting edge of our disciplines and of interest to our readership.

Creating the most cited journal? Creating cutting edge? This is a pre-internet 1980’s attitude of  a journal editor trying to get a higher citation impact in the competition with other journals. It simply devalues everything that Derickson does not understand or that Derickson does not want to promote.

So my initial enthusiasm of the paper finally dies with “the biggest scientific experiment”

Huge interventions should have huge effects. If you drop $100 million on a school system, for instance, hopefully it will be clear in the end that you made students better off. If you show up a few years later and you’re like, “hey so how did my $100 million help this school system” and everybody’s like “uhh well we’re not sure …

Yes, this is about the end of scholarly peer review as peer review fails  to catch major errors in about 1/3 of all papers.

In all sorts of different fields, research productivity has been flat or declining for decades, and peer review doesn’t seem to have changed that trend. New ideas are failing to displace older ones. Many peer-reviewed findings don’t replicate, and most of them may be straight-up false. When you ask scientists to rate 20th century discoveries that won Nobel Prizes, they say the ones that came out before peer review are just as good or even better than the ones that came out afterward.

The focus is on “cutting edge” and “interest” aka impact points but  neither on ingenious minds nor brilliant discoveries.

The Absurdity of Peer Review. What the pandemic revealed about… | by Mark Humphries | Jun, 2021 | Elemental June 2021

I was reading my umpteenth news story about Covid-19 science, a story about the latest research into how to make indoor spaces safe from infection, about whether cleaning surfaces or changing the air was more important. And it was bothering me. Not because it was dull (which, of course, it was: there are precious few ways to make air filtration and air pumps edge-of-the-seat stuff). But because of the way it treated the science.
You see, much of the research it reported was in the form of pre-prints, papers shared by researchers on the internet before they are submitted to a scientific journal. And every mention of one of these pre-prints was immediately followed by the disclaimer that it had not yet been peer reviewed. As though to convey to the reader that the research therein, the research plastered all over the story, was somehow of less worth, less value, less meaning than the research in a published paper, a paper that had passed peer review.

I expect the business of scientific publishers is slowly coming to an end. Maybe others also?

The solution to scientific publishing might be going back to writing books instead of papers. Can’t decide if I’m joking. There’re too many papers published, books will be of higher quality (take time & effort). Urgent results could be pre-printed & discussed in conferences.

— Oded Rechavi (@OdedRechavi) October 31, 2021

We will need of course peer evaluation but maybe not in the sense that scientific publication is being suppressed by peer review of some elite journals. Some arXiv type PDF deposit plus some elife/twitter/pubpeer score would be fully sufficient. For me and maybe also for many other people in the field.