A new program from the Science Exchange, in partnership with PLOS and Figshare, aims to tackle a growing issue in the scientific community: published results that can't be replicated in another lab.
The new effort, called the Reproducibility Initiative, sets scientists up with research facilities to conduct independent validation studies on a fee-for-service basis. Validated studies receive a certificate of reproducibility; researchers can publish the replicated results in a special collection in PLOS One; and the validated data and results are made available through Figshare's open-access repository.
Elizabeth Iorns, Science Exchange's co-founder and CEO, estimates that up to 70 percent of research from academic labs cannot be reproduced, "representing an enormous waste of money and effort." She adds that the problem has primarily been "in the lack of incentives and opportunities for validation" and says the Reproducibility Initiative "directly tackles these missing pieces."
"I very much hope that this catches on," says Derek Lowe at In the Pipeline. "I can especially see this being used when someone is planning to commercialize some new finding — going to the venture capital folks with independent verification will surely count for a lot. Granting agencies should also pay attention, and reward people accordingly."
Commenters at Lowe's blog wonder about the economics of the proposal, however. "The average study that I publish probably costs >$200K if labor is included, take[s] at least a year to perform, and require[s] highly specialized facilities. I can't see me paying someone else to reproduce such studies any time soon," says bacillus. Boghog notes that "this could really take off if the funding agencies directly paid for such studies."
Over at Slate, Carl Zimmer also raises the question of cost, noting that it "will probably fare best when the science involved can be easily duplicated with standard pieces of technology. The more cutting edge the research, the fewer people will be able to replicate it."
As an example, he cites the recent study from Jay Shendure and colleagues in which they sequenced the genome of a fetus using blood from the mother and saliva from the father, which is "a tour de force of DNA isolation and computer analysis — one that only a few labs in the world today could manage."