Following up a recent PubPeer post the journals integrity officer Dawn Peters wrote to me „You may submit a Perspective or letter to the editor“. So I wrote this letter.
NEJM papers forming the empirical backbone of the hygiene hypothesis contain important methodological weaknesses. The journal’s role was active rather than passive: the Bach 2002 review (1) and the accompanying editorial (2) were clear endorsements published alongside the first prominent farm/endotoxin findings. The hypothesis was rarely framed to allow falsification, and was repeatedly reformulated – from “infections” to “endotoxin” to “microbial diversity” to “innate immunity.” Later disclosures of editorial conflicts of interest make a retrospective methodological audit overdue.
Bach (1), cited nearly 3,700 times, built its central argument on figures whose source data cannot be verified. Figure 1A cites a source containing seroprevalence rather than incidence data. Figure 1B assigns incorrect country categories. Figure 3 combines disease and economic data from sources that do not contain the values shown. Figure 4 relies on an unpublished personal communication that has not been replicated. The ecological framing throughout is insufficient to support causal inference.
Braun-Fahrländer (3) pools farming and non-farming children whose endotoxin exposures differ twofold into a single smoothed curve. The smoothing span is changed selectively for the one outcome contrary to the main hypothesis, without justification. Fewer than one third of eligible participants provided complete data, with no analysis of non-completers. After correction for multiple testing, virtually the entire Table 2 collapses to a single marginal result.
Ege (4) excluded wheeze-enriched children from the PARSIFAL sample, reducing wheeze prevalence from 8% to 3%, without disclosure in the main paper. In the resulting sample, the farm-asthma association is non-significant. The SSCP normalization standard contained the same organisms highlighted as the paper’s headline protective finding. In the paper’s own final model, the GABRIELA diversity result is null (OR 1.01, p=0.93) – neither value reported in the abstract.
Stein (5) infers genetic equivalence between Amish and Hutterite children from principal-component analysis of common SNPs, a method not suited to detecting the rare founder variants that distinguish these populations. Shared ancestry is thus not established, and the paper’s central contrast – attributing the four-fold asthma difference to farming environment rather than genetic background – is not warranted.
These concerns – unverifiable source data, undisclosed sample exclusions, selective analytical choices, and abstracts that omit null results from the papers’ own final models – are documented on PubPeer and remain unaddressed. Taken together, they indicate that the hygiene hypothesis was not established on sound empirical foundations.
I now received this response
Dear Prof. Wjst:
I am writing about your recent letter to the editor. We sent your concerns to authors of the studies you referenced and reviewed the replies we received as well as the studies themselves. I am sorry to say that your letter was not accepted for publication. We believe that limitations you raise were adequately acknowledged by the authors in the published papers and/or were consistent with reporting practices at the time of publication.
Thank you for the opportunity to consider your letter.
Sincerely,
Eric Rubin, MD, PhD
Editor in -Chief
So the New England Journal of Medicine has resolved my concerns about five hygiene hypothesis papers by consulting the authors. One is deceased. The rest are retired. All confirmed their work was fine, a somewhat predictable outcome.
The NEJM calls this research integrity. I call it a new normal: where the bar for correcting the scientific record is the posthumous approval of those who created it. The new normal – figures citing the wrong sources, null results missing from abstracts, undisclosed sample exclusions and a key figure that rests on an unpublished personal communication that has never been replicated. This is an interesting new benchmark for a journal of the NEJM’s standing.
One for the files.
