The Drosophila melanogaster Genetic Reference Panel
Mackay, Richards et al., Nature
North Carolina State University's Trudy Mackay and her colleagues present the Drosophila melanogaster Genetic Reference Panel, "a community resource for analysis of population genomics and quantitative traits."
Significant, But Not Plausible
March 04, 2010
You can't trust medical studies -- or at least the statistics, said a panel at the American Association for the Advancement of Science according to Ars Technica. A major issue is the sheer number of data points and the number of tests that researchers are performing. "Even given a low tolerance for error, the sheer number of tests performed ensures that some of them will produce erroneous results at random," the post says. In addition, models can account for the same factor through different mathematical approaches and then spit out completely different results. Members of the panel said that funding agencies and journal editors should lobby for the use of new statistical methods and hold studies to "minimal standard of biological plausibility."
"Lobby for the use of new
"Lobby for the use of new statistical methods"???
My question is, Why is "new" always the solution?
How many "new statistical methods" have no verification whatsoever, but are just the test-du-jour (especially in molecular evolution), where everybody is free to invent their own test statistic. Maybe we could learn how to use the "old" methods properly? Or ask scientists to collaborate with those who DO understand statistical methods, eg, at a statistical consulting lab. Much of the problem discussed in that post above concerns the failure to correct for multiple comparison, like the Bonferroni technique, and replication techniques, like the bootstrap and cross-validation. But it may be too much to ask in an era when anyone's monkey's uncle can push a button on a statistical package, when academic tenure is based having on lots of publications (true or not), and when corporate profits are based on the ability to demonstrate a significant effect (real or not). As they say in the article, "we simply can't rely on the researchers to do it." I think placing the onus on journal editorial policy and funding agencies maybe makes sense.