There Goes the Significance

Sometimes after seeing an initial interesting result from a project, that trend or its significance peters off in replication studies, leading a New Yorker article from Jonah Lehrer to wonder if there is something wrong with the scientific method. The decline, Lehrer says, is partially explained by publication bias or selective reporting, but the rest of the explanation may be due to randomness or noise. "This suggests that the decline effect is a decline of illusion," Lehrer says. He cites discrepancies in the benefits of anti-psychotics to even some seen in measuring gravity and says that "such anomalies demonstrate the slipperiness of empiricism."

HT: Razib Khan at Gene Expression