Schlagwort-Archive: statistics

The data may not contain the answer

(stolen from )

John Tukey (1915-2000), American mathematician best known for the Fast Fourier Transform (FFT) algorithm and the box plot

Cause and effect in observational data: Magic, alchemy or just a new statistical tool?

Slashdot has a feature on that

Statisticians have long thought it impossible to tell cause and effect apart using observational data. The problem is to take two sets of measurements that are correlated, say X and Y, and to find out if X caused Y or Y caused X. That’s straightforward with a controlled experiment… But in the last couple of years, statisticians have developed a technique that can tease apart cause and effect from the observational data alone. It is based on the idea that any set of measurements always contain noise. However, the noise in the cause variable can influence the effect but not the other way round. So the noise in the effect dataset is always more complex than the noise in the cause dataset. .. The results suggest that the additive noise model can tease apart cause and effect correctly in up to 80 per cent of the cases (provided there are no confounding factors or selection effects).

and jmlr a more theoretical account

Based on these deliberations we propose an efficient new algorithm that is able to dis- tinguish between cause and effect for a finite sample of discrete variables.