Being able to reproduce research results is getting harder all the time, writes George Johnson at his Raw Data column in the New York Times.
John Ioannidis argued back in 2005 in PLOS Medicine that most scientific hypotheses were likely to be wrong at the outset and that subtle biases influence how researchers interpret their data, especially as competition for grant intensifies. He also, Johnson notes, examined a decade's worth of papers and found that many of their findings were contradicted by other papers, a finding Ioannidis reported in the Journal of the American Medical Association in 2005.
Similarly, Johnson notes, Glenn Begley, the chief scientific officer at TetraLogic Pharmaceuticals, wrote in Nature in 2012 that when he was at Amgen he and his colleagues couldn't reproduce the results of some 47 out of 53 cancer studies, even with help from the researchers who worked on the original studies.
Johnson adds that the issue of reproducibility is going to worsen as more and more data is produced. "Exciting new results will continue to appear. But as the quarry becomes more elusive, the trophies are bound to be fewer and fewer," Johnson says. "If a result appears only under the full moon with Venus in retrograde, is it truly an advance in human knowledge?"