Peer review is often criticized as being broken, particularly in times of tight grant funding and increased applications for that funding, note Michael Lauer and Richard Nakamura, respectively from the Office of Extramural Research and the Center for Scientific Review at the US National Institutes of Health.
In the New England Journal of Medicine, the duo notes that there have been calls to review peer review itself.
"Since review scores are seen as the proximate cause of a research project's failure to obtain support, peer review has come under increasing criticism for its purported weakness in prioritizing the research that will have the most impact," they write.
They add, though, that novel approaches are likely needed to measure scientific impact within large, national funding programs.
Still, they say that better statistics could be identified to help interpret application review scores. Study sections could, for instance, report percentile rankings along with confidence ranges, and "range reporting" or "score binning" could enable staff to weigh investigators' track records and their other support as well as the diversity of the institute's portfolio and its strategic goals.
Further, Lauer and Nakamura say that studies of anonymous peer review could also uncover how factors like investigators' age, institution, sex, and race or ethnic background influence the approval and impact of research projects.
They add that "we at the NIH are eager to work with the extramural research community to foster and enhance a scientific culture surrounding our work as funders and overseers of peer review."