How often does established research get overturned by a newer, more persuasive study, asks the Guardian's Ben Goldacre. How many times does new work show that old methods are ineffective? A study published in the Archives of Internal Medicine looks at 212 academic papers published in the New England Journal of Medicine in 2009 to assess whether the new findings published in each overturned previous research. Of those papers, 124 made a claim about whether a treatment worked or not, though 73 were new claims and so had nothing to overturn. Of the remaining 51, Goldacre says, 16 upheld a current practice as beneficial, while 16 others found that current practice was ineffective. The remaining 19 were inconclusive. "This looks like a reasonably healthy state of affairs: there probably are true tales of dodgy peer reviewers delaying publication of findings they don't like, but overall, things are routinely proven to be wrong in academic journals," Goldacre says. "Equally, the other side of this coin is not to be neglected: we often turn out to be wrong, even with giant, classic papers. So it pays to be cautious with dramatic new findings; if you blink you might miss a refutation, and there's never an excuse to stop monitoring outcomes."
'Research About Research'
Jul 18, 2011