–Day 3 of Just Science Week–

Peer review certainly plays a major role in assuring quality of science. There are many positive aspects of peer review (plus a few disadvantages like promoting mainstream). Systematic research on peer review, however, has been largely absent until 2 decades ago; after 5 international conferences on peer review there is now also the WAME association of journal editors. Over the years, I have experienced the “cumulative wisdom” thrown at my own papers and of course developed my own style when doing reviews. Last week PLOS medicine published an interesting study who makes a good peer review:

These reviewers had done 2,856 reviews of 1,484 separate manuscripts during a four-year study period, and during this time the quality of the reviews had been rated by the journal’s editors. Surprisingly, most variables, including academic rank, formal training in critical appraisal or statistics, or status as principal investigator of a grant, failed to predict performance of higher-quality reviews. The only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth (under ten years of experience after finishing training), and even these were only weak predictors.

The first finding may be unimportant for non-medics but the second may apply to a larger audience. What I fear – and that is usually not mentioned in the current discussion – that the peer review system is slowly suffocating. The willingness to do this (unpaid & extra) work is going down as papers (at least in my field) are produced more and more an industrial mass production level. I am getting a review request nearly every second day while I do need between 30 minutes and 3 hours for a paper. So, less is more.

Addendum

For a follow up go to sciencesque, a scenario how science in the post-review phase will work.