The Hits Just Keep on Coming

In the Pipeline's Derek Lowe says there are several reasons why the credibility of the scientific literature has been called into question lately, and it has to do with factors both in and out of the research community. In the US, the left/right political divide is a problem that's getting bigger, with cultural and policy disagreements often getting entangled with scientific issues like evolution and climate change. Then there's all the attention being paid to retractions — "suspicions of commercial bias in the medical literature, the problems of reproducibility of cutting-edge results, and to round it all off, several well-publicized cases of fraud," Lowe says. Even without the political "axe-grinding" there's still a lot of concern over whether the literature is accurate. And then there's the misuse of statistics. Lowe points to a Guardian piece that says that too many papers in the field of neuroscience are using improper statistics when comparing "differences-between-differences." Just because something is statistically significant under a given set of conditions, that doesn't mean it'll be statistically significant under a second set of condition, and usually a researcher needs to do more work to see if that's the case. But too many people are making the comparisons without doing the work, the Guardian article says. "So the questions about the reliability of the literature are going to continue, with things like this to keep everyone slapping their foreheads," Lowe says. "One can hope that we'll end up with better, more reliable publications when all this is over. But will it ever really be over?"