Research reproducibility is a big concern these days, but a trio of Stanford University researchers wonders in Science Translational Medicine what people actually mean when they talk about reproducibility.
Does reproducibility mean being able to replicate a study and its results using the same materials or with a separate dataset? Or does it mean that the study was reliable, Stanford's John Ioannidis and his colleagues ask. "The fundamental concern … is, in fact, not reproducibility per se, but whether scientific claims based on scientific results are true," they write.
He and his colleagues then propose new terms to describe the various facets of reproducibility. In their lexicon, 'methods reproducibility' refers to the ability to re-conduct an experiment with the same data and tools to get the same results; 'results reproducibility' means being able to replicate the study independently to produce corroborating results; and 'inferential reproducibility' refers to drawing similar conclusions from a study replication or re-analysis.
At Stat News, Ivan Oransky and Adam Marcus from Retraction Watch note that Ioannidis and his colleagues don't weigh in on whether or not there is a reproducibility crisis, but do say researchers need to better share their data, methods, and other information that's important for the experimental process.
They also laud Ioannidis and his colleagues' attempt to clarify just what is under discussion. "In the tale of Chicken Little, reasonable farm animals could disagree about whether the sky was falling, but no one had any misconceptions about what the petite poulet was chirping about," Oransky and Marcus write. "The discussion of reproducibility needs its own lingua franca."