Science has been facing a bit of a crisis as some recent studies have indicated that the findings from many papers can't be reproduced. For instance, Amgen researchers have reported that they weren't able to reproduce the results of some 47 out of 53 landmark cancer papers, while in psychology, the Open Science Collaboration had reported that it could reproduce only about half of the nearly 100 major studies in the field it attempted.
But that latter finding is now itself in dispute. In a technical comment appearing in Science this week, researchers from Harvard University and the University of Virginia say that it is marked by three statistical errors.
"Our analysis completely invalidates the pessimistic conclusions that many have drawn from this landmark study," Harvard's Daniel Gilbert tells Nature News.
In particular, he and his colleagues say that the replication studies often drew upon different populations than the original studies did; the replication attempts were underpowered; and deviations from the original protocols might have introduced bias. Instead, after accounting for those, Gilbert and his colleagues say the paper should've come to the opposite conclusion: psychology studies are reproducible.
The original authors led by Brian Nosek from the Center for Open Science though, respond that Gilbert and his colleagues' analysis, too, suffers from statistical issues and assumptions.
"The dispute all at once speaks to why we need more reproducibility in science and questions whether there really is a reproducibility crisis at all," the Verge adds.