Small reviewer biases can have a large effect on grant application outcomes, according to a new study available online at Research Policy.
The Children's Hospital of Philadelphia's Theodore Eugene Day developed a computer simulation of the peer review process and generated a set of 2,000 grant applications, half from preferred class and half from non-preferred class investigators. Three computer-generated reviewers then scored each grant, and those scores together with the funding line determined whether or not the application was funded.
While Day assigned an intrinsic quality to each of the simulated applications, he also introduced randomness into the reviewers' scores to better match real-world variations. The intrinsic quality of the simulated applications was largely matched between the groups.
He also then varied the number of reviewers that exhibited bias and the extent of that bias. When the bias reduced the scores of non-preferred investigators by 2.8 percent, Day began to see statistically significant differences in funding decisions. A 3.7 percent reduction in scores for non-preferred investigators led to the funding of 118 preferred-class investigators and 82 non preferred-class investigators.
ScienceInsider notes that this boils down to non-preferred investigators having to submit better quality grants to receive funding, while preferred investigators can get lower quality grants funded.
"It shows how systemic bias against any group translates into fewer dollars and cents to [a scientist] belonging to that group, irrespective of other factors," the University of Hawaii, Manoa's Hope Jahren tells ScienceInsider.