The title is taken from an essay of Dan Bolnick about current PubPeer practices
That said, there is some question about the proper procedure for answering these criticisms. Yes, PubPeer itself leaves room for comments (interestingly, journal editors like myself must pay money to reply to comments, even if to acknowledge them and state we are evaluating the issue). But, this process bypasses the journal that publishes the paper, and bypasses the normal scientific tradition of external review by experts in the field chosen by the journal editor for their knowledge and hopefully objectivity. For this reason, I want to really encourage people with substantial concerns about a paper (e.g., which may appreciably alter the results and conclusions), to submit formal “Comments” (different journals call these different things) to the journal.
I agree, letters or comments would be preferable. In practice, however, comments are largely ignored. I can provide numerous examples where nothing happened.
But, scientific traditions are fluid and we are in an era of increasing speed and openness: Preprint servers, open peer review, open data, et cetera. We therefore also recognize that PubPeer is an active tool in science conversations. The criticisms posted there can be valid identification of genuine problems that need to be evaluated formally and corrected. If valid well-justified and substantial concerns exist and are published on PubPeer, then the affected journal should respond.
Yea, yea. And are there caveats? Bolnick thinks
it is important that these not be used as a mechanism for pursuing personal vendettas. Excessive targeting of an author with multiple minor complaints can constitute a kind of harassment, and may be viewed as such by University Equity officers or equivalent. The anonymous nature of many PubPeer comments makes it easier for impacted authors to feel like (and, argue that) they are the target of personal vendettas and harassment. Second, the existence of PubPeer comments can cast a long shadow over a paper whether the comments are profound or minor. This shadow can affect an author’s career prospects (fellowship applications, job applications, etc) even before the matter is resolved and judged to be valid or not. The result can be inappropriate damage to an innocent authors’ career, which in turn may have grave consequences for mental health. Third, there is an established mechanism for voicing complaints about papers in science: Contact the author to request clarification, or contact the Editor, or submit a Comment.
Well, it depends. What is a minor complaint? One wrong label, two or three? Mixing up figures and data? Nirwana references? Intentional wrong statements? Ignorance of literature? If a comment is profound or minor can be decided by any PubPeer reader. Grave consequences for mental health of a fraudulent or careless author?
I believe that if there would be any working mechanism for voicing complaints about scientific integrity (or just minor corrections), PubPeer would not exist. Kudos Brandon Stell.