The first studies out of the Reproducibility Project: Cancer Biology have mixed results, Retraction Watch reports.
The project was launched a few years ago by eLife, the Center for Open Science, and Science Exchange and aimed to replicate key findings from 50 high-profile preclinical cancer studies. The project, though, has had to scale back its plans because of time and cost considerations. RW's Ivan Oransky and Adam Marcus note at Stat News that each replication effort took an average seven months and cost an average $27,000.
The project this week published in eLife its efforts to reproduce five studies. At In the Pipeline, Derek Lowe says it was able to replicate two studies fairly well, though those replication efforts suffered from their own issues. Replication of two other studies couldn't be interpreted in comparison to the original work and the final one seems to not have been able to be replicated. The author of the non-reproduced paper tells Nature News that other labs have said they'd been able to replicate that work.
And that, Nature News notes, worries some of the original researchers. "Careers are on the line here if this comes out the wrong way," Atul Butte from the University of California, San Francisco. His Science Translational Medicine paper was "mostly substantiated" by the reproducibility effort.
Lowe adds that the reproducibility teams weren't able to do exactly what the original researchers did and that these papers highlight how difficult it is to reproduce papers. "[The project is] already telling us that the literature in the field is probably inadequate in many respects and that there are a lot more factors at work than you might have guessed from reading the original papers," he says. "It's good to have proof of that, but honestly, we knew it already."