hype | Science Surf

We always laughed at the papers in the “Journal of Irreproducible Results”

https://www.thriftbooks.com/w/the-best-of-the-journal-of-irreproducible-results/473440/item/276126/?gclid=EAIaIQobChMI3NnCm72l-QIVpHNvBB1nIwSWEAQYAiABEgK6__D_BwE#idiq=276126&edition=1874246

then we had the replication crisis and nobody laughed anymore.

And today? It seems that irreproducible research is set to reach a new height. Elizabeth Gibney discusses an arXiv paper by Sayash Kapoor and Arvind Narayanan basically saying that

reviewers do not have the time to scrutinize these models, so academia currently lacks mechanisms to root out irreproducible papers, he says. Kapoor and his co-author Arvind Narayanan created guidelines for scientists to avoid such pitfalls, including an explicit checklist to submit with each paper … The failures are not the fault of any individual researcher, he adds. Instead, a combination of hype around AI and inadequate checks and balances is to blame.

Algorithms being stuck on shortcuts that don’t always hold has been discussed here earlier . Also data leakage (good old confounding) due to proxy variables seems to be also a common issue.

Most recently, a NEJM paper “Vitamin D Deficiency — Is There Really a Pandemic?” by Manson, Brannon, Rosen, and Taylor explains the big misunderstandings that let some authors conclude that whole populations are being vitamin D deficient. Just to recall, the IOM recommended in 2010 serum concentrations of vitamin D (i.e., 25-hydroxyvitamin D [25(OH)D]) above 20 ng per milliliter (or 50 nmol per liter) as appropriate level and supplementation with 600 to 800 IU per day as Recommended Dietary Allowance (RDA). And here are the 8 facts: Continue reading The end of the vitamin D deficiency debate? 8 facts →

Tag Archives: hype

(replication crisis)^2

The end of the vitamin D deficiency debate? 8 facts