Tag Archives: sleuth

Scientific integrity investigators are the last resort

It's basically the same scene – political journalists are fact checking interviews afterwards. OSINT experts are verifying satellite and ground images. Science sleuths are verifying images and tables in scientific publications. The stakes are high here as integrity investigators are the last resort. It is a bit like the old "sub umbra dei" saying, there is only god behind us. There is no room for error.

In the science integrity scene there are many mavericks or let's say colorful people. Most come from inside academia, starting or ending a career there. Some are from far outside it, and their motives are not hard to guess ranging from methodological curiosity to personal grievance. What holds the field together is therefore not a shared background but a shared refusal to accept a published claim on authority of a journal alone. That heterogeneity is a strength for detection but also a weakness for credibility, because a single overreaching claim by one investigator is readily used to discredit the rest as "Pubpeer Mob". The stakes are high: A minor error in a published paper costs the author a correction. An unfounded allegation can cost a career, and it costs the field its licence to look. That asymmetry sets the working rules. No ten commandments but some working rules.

Methodological rigour comes first: every claim must be stated so that a third party can check it against the same source material and arrive at the same result. Image duplications get exact border coordinates, statistical objections need the extracted values on which a test was run, because an eyeballed anomaly is not a finding. Distribution checks only carry weight when the reported summary statistics, rounding conventions and sample sizes are documented well enough for someone else to replicate them. Replication is key, also here.

Relevance is also a filter as much as rigour is. Typographical slips, reference-list errors, image panel mix up and stylistic issues dilute the serious concerns and shift the discussion from the science to the tone of the critic. These errors are annoying and do not add to the credibility of authors. What really matters is whether a defect could change the conclusions: fabricated or duplicated data, misinterpreted assays and conditions, results incompatible with the stated design, analyses that cannot have produced the reported numbers, an well, undisclosed conflicts that are directly related on interpretation. These are the high scores not the 18,000 minor issues produced by some software.

Everything else stays out. Statements about what the authors intended, whether they are competent, or what a pattern of errors implies are not verifiable. Even subtle threats are inappropriate, even jokes and sarcastic comments are not wanted, I had to learn. The finding that some numbers do not reconcile, that images have been used before, that the trial registration postdates the enrolment: this is the relevant stuff. Whether this arose from fraud, sloppiness or software error is for the institution or the editorial staff to verify, not for the sleuth writing the comment. Only journals or institutions can ask questions that must be answered; the sleuth can document, but cannot compel a response, he has no access to raw data, neither can he interview a senior coauthor.

That limitation is worth holding on up, because it defines what a post-publication review can honestly claim: the published record, as it stands, does not add up. It can classify the level of the integrity pyramid but not claim to know why an issue occured. Fervour is the occupational hazard here, and it is super expensive: the investigator who is right about ninety-nine findings and overstates the hundredth hands every previously criticised author a reason to dismiss the rest.

Restraint is not politeness, it is the condition under which the work retains its force.

 

CC-BY-NC Science Surf , accessed 04.09.2026