Clinical trials using animal models often generate positive significant results that are not easy to replicate in human trials. That is because many of these studies are biased to generate positive results, says Stanford University Professor John Ioannidis.
Heidi Ledford writes in Nature that Ioannidis, who has made a name publicizing flaws and failures in scientific research models, recently led a project which found that around 40 percent of animal studies of neurological diseases reported finding statistically significant results, which he says is too many.
His analysis of 4,000 data sets found that statistically significant results were reported nearly twice as much as they should be, when compared against the results of a single large study.
“The results are too good to be true,” he says, which could partly explain why many therapies that perform well in preclinical studies often are not successful in human trials.
Ioannidis and his partners reviewed 160 meta-analyses of over 4,400 studies of Alzheimer's and Parkinson's diseases, as well as spinal cord injuries and other disorders, and predicted that 919 of those should generate statistically significant findings. But they found that 1,719 of these studies reported significant findings. They also found that "an inflated number of significant findings" were most likely to come from studies with the smallest sample sizes, and from projects with authors who report a financial conflict of interest.
Ioannidis says the findings do not mean that animal studies are meaningless, Ledford writes, but that they should have better controls and reporting standards. He also advocates for a registry of animal studies that would publicize negative findings and research protocols.