{"id":26284,"date":"2026-04-13T22:44:43","date_gmt":"2026-04-13T20:44:43","guid":{"rendered":"https:\/\/www.wjst.de\/blog\/?p=26284"},"modified":"2026-04-14T16:51:21","modified_gmt":"2026-04-14T14:51:21","slug":"why-you-cant-change-the-rules-after-the-game-has-started","status":"publish","type":"post","link":"https:\/\/www.wjst.de\/blog\/sciencesurf\/2026\/04\/why-you-cant-change-the-rules-after-the-game-has-started\/","title":{"rendered":"Why you can\u2019t change the rules after the game has started"},"content":{"rendered":"

It was a long wait – 10 years<\/a> –\u00a0 for the Vitality vitamin D study in newborns to come to an end. It is super disappointing to see now their first study abstract<\/a> at the 2026 AAAAI Annual Meeting with a null result. Besides the fact that they got it wrong – Vitamin D3 supplementation was never protective but allergy risk\u00a0 in newborn<\/a> –\u00a0 the following \u00a0three screenshots show something that should make any methodologist uncomfortable.<\/p>\n

The VITALITY trial filed its first ClinicalTrials.gov record<\/a> in April 2014. By November 2022 with the data at hand, version 12 shows sweeping rewrites to both the primary outcome and the inclusion criteria. Pink = deleted, green = added.<\/p>\n

\"\"<\/a> \"\"<\/a> \"\"<\/a><\/p>\n

What changed, concretely:<\/p>\n

– The original primary outcome was \u201cprevalence and severity of challenge-proven food allergy in participants with positive skin prick tests at age 12 months.\u201d Version 12 splits this into two primaries, adds a second time point (6 years of age), and quietly drops \u201cseverity\u201d and the SPT-positive filter. That filter was doing real work: restricting the analysis to sensitised children would have given a much smaller, higher-risk denominator.
\n– The inclusion criteria went from \u201chealthy, term, breastfeeding infants\u201d to a detailed specification with an age window (6\u201312 weeks), formula tolerance up to 120 mL\/day, and a new informed-consent bullet. Each addition narrows or shifts the enrolled population.<\/p>\n

\"\"<\/a>
Reporting two significant p-values from what are effectively subgroup analyses in an abstract whose primary result failed (p=0.537) is textbook outcome fishing. With 10 allergens tested, finding two below 0.05 by chance alone is entirely expected. In short: a failed trial has been dressed as a promising one through a combination of dual estimands, allergen subgroup mining, heavy imputation, and conclusion spin. Each element is individually defensible in isolation; together they constitute a coordinated rhetorical strategy to salvage a null result.<\/figcaption><\/figure>\n

Why this is a fundamental problem?<\/em><\/p>\n

Clinical trials are hypothesis tests, not explorations. The logic is identical to a one-sample t-test: you fix the null hypothesis, the test statistic, and the decision threshold before you look at the data, because the Type I error rate (your false-positive probability) is only valid under those pre-specified conditions. The moment you select or redefine your outcome after seeing interim results \u2014 even partially, even innocently \u2014 you are performing an implicit multiple comparison. You have, in effect, tested several hypotheses and reported only the one that worked.<\/p>\n

Changing inclusion criteria mid-study is equally damaging. It redefines the population to which your result generalises. If the original enrolment targeted a broader group and the amended criteria select a more compliant or biologically distinct subgroup, the treatment effect you ultimately report belongs to a population that was never pre-specified. Reviewers and readers have no way to know whether the amendment was scientifically motivated or outcome-motivated.<\/p>\n

The specific mischief of outcome switching<\/em><\/p>\n

Dropping \u201cseverity\u201d from the primary outcome is not cosmetic. A trial that fails on prevalence-plus-severity can be reframed as a success on prevalence alone. Dropping the SPT-positive filter expands the denominator, which typically dilutes an effect \u2014 unless the intervention actually works better in unselected infants, a hypothesis that was apparently not the original one. Adding a six-year follow-up endpoint transforms a 12-month study into something else entirely, with a different sample-size justification and a different regulatory profile.<\/p>\n

What preregistration is supposed to prevent<\/em><\/p>\n

The entire point of a trial registry is to create a timestamped public contract. Investigators declare in advance: this is our question, this is our population, this is how we will measure success. Journals and regulators can then verify that the published analysis matches the contract. When version 12 diverges this substantially from version 2, the contract has been renegotiated \u2014 and the renegotiation happened after years of data collection, when results were at least partially visible to the investigators.<\/p>\n

This does not automatically mean misconduct. Trials genuinely need protocol amendments \u2014 safety signals emerge, recruitment proves impossible under original criteria, regulatory agencies request changes. But every such amendment requires a documented, dated rationale filed before the analysis is run, and the published paper must report both the original and amended specifications with transparent explanation. Silently absorbing eight versions of changes into a final paper, with no mention of what the original primary endpoint was, converts a confirmatory trial into a disguised exploratory one \u2014 while retaining the inferential authority of a pre-registered RCT.<\/p>\n

The VITALITY screenshots are a clean teaching example of exactly this problem.\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b<\/p>\n\n

 <\/p>\n

\n CC-BY-NC Science Surf , accessed 14.04.2026<\/span>\n <\/div>","protected":false},"excerpt":{"rendered":"

It was a long wait – 10 years –\u00a0 for the Vitality vitamin D study in newborns to come to an end. It is super disappointing to see now their first study abstract at the 2026 AAAAI Annual Meeting with a null result. Besides the fact that they got it wrong – Vitamin D3 supplementation … Continue reading Why you can\u2019t change the rules after the game has started<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,10],"tags":[5108,5106,5107],"class_list":["post-26284","post","type-post","status-publish","format-standard","hentry","category-asthma-allergy","category-sunshine-vitamin","tag-katrina-allen","tag-kirsten-perrett","tag-shyamali-dharmage"],"_links":{"self":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts\/26284","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/comments?post=26284"}],"version-history":[{"count":8,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts\/26284\/revisions"}],"predecessor-version":[{"id":26298,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/posts\/26284\/revisions\/26298"}],"wp:attachment":[{"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/media?parent=26284"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/categories?post=26284"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.wjst.de\/blog\/wp-json\/wp\/v2\/tags?post=26284"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}