The inability to reproduce research findings is a long-standing issue in the sciences, and a new paper appearing in PLOS Biology estimates that some $28 billion is spent each year on preclinical research that's not reproducible.
A trio of researchers from the Global Biological Standards Institute and Boston University School of Management calculated that more than half of preclinical research isn't reproducible. They came up with this number by analyzing the four basic causes of irreproducibility: study design, biological reagents and reference materials, lab protocols, and data analysis and reporting. Based on past published error rates in those categories, errors in each of these categories lead all together to a between 18 percent and 88.5 percent irreproducibility rate, according to the researchers' probability bounds approach. The midpoint of that range is about 53 percent.
"That seems high to me, and I'm fairly cynical about the literature," notes Derek Lowe at In the Pipeline. "But they've based this estimate on the many publications that have addressed problems with antibodies, with study designs, with contaminated cell lines, and many other factors, and if you start adding all these things up, the numbers get pretty alarming."
And that 53 percent works out to be about $28 billion. NPR notes that the researchers caution that not all of that is money wasted. Some of these studies aren't reproducible because the methods were poorly described, not because the results aren't valid.
Still, the researchers tell NPR that some fixes like better study-design training for investigators and cell-line testing could bring that cost down.