The greatest bias in the scientific literature comes from small studies that report large effect sizes, according to Stanford University's John Ioannidis and his colleagues in the Proceedings of the National Academy of Sciences this week.
Ioannidis and his colleagues sifted through more then 3,000 meta-analyses from 22 different scientific fields, searching for patterns of bias. In particular, they focused on seven types of bias, including the small-study effect in which studies with small sample sizes report large effect sizes; the early-extremes effect in which provocative findings are published early; and citation bias where studies with larger effect sizes are more likely to be cited.
"I think that this is a mapping exercise," says Ioannidis in a statement. "It maps all the main biases that have been proposed across all 22 scientific disciplines. Now we have a map for each scientific discipline, which biases are important and which have a bigger impact, and therefore scientists can think about where do they want to go next with their field."
As Ioannidis and his colleagues report in PNAS, their "bird's-eye view" of bias in the scientific literature found that the greatest source of bias was small studies. They additionally found that early-career researchers, isolated scientists, and investigators with a history of misconduct were more likely to overestimate effect size.
Retraction Watch notes with surprise that the analysis didn't find an effect stemming from the pressure to publish.Â