In an opinion column in Nature this week, McGill University neuroscientist Jeffrey Mogil and University of Edinburgh neuroscientist Malcolm Macleod write that solving the "reproducibility crisis" in scientific papers may require a new kind of paper altogether — something that would allow researchers to trust the conclusions they read in other papers and would also allow basic scientists to "explore and innovate."
They propose a paper "that incorporates an independent, statistically rigorous confirmation of a researcher's central hypothesis. We call this large confirmatory study a preclinical trial. These would be more formal and rigorous than the typical preclinical testing conducted in academic labs, and would adopt many practices of a clinical trial."
This would make researchers take a second look at their own data, Mogil and Macleod write, which would engender more confidence in the results.
"The confirmatory study would have three features," they write. "First, it would adhere to the highest levels of rigour in design (such as blinding and randomization), analysis and reporting. Second, it would be held to a higher threshold of statistical significance, such as using P values of P < 0.01 instead of the currently standard P < 0.05. Third, it would be performed by an independent laboratory or consortium."
This would be a big change in the thinking behind scientific papers, but science would benefit, the authors suggest. It would also make exploratory science fun again and would decrease the instances of fraud, they predict.
And although this proposal wouldn't fix all the problems inherent in the scientific publication system, "we believe it is worth a try," they write.