Percentile rankings given to grants seeking US National Institutes of Health funding are not good predictors of citation impact, according to a recent study published in Circulation Research.
Investigators led by Michael Lauer from the National Heart, Lung, and Blood Institute examined nearly 1,500 R01 grant applications funded by NHLBI between 2001 and 2008, and their ensuing publications and citations. Grants with a percentile ranking of less than 10 percent had a median eight publications and an H-index of six; grants falling in the 10 percent to just under 20 percent range also had a median eight publications and an H-index of six; and grants falling in the 20 percent to 41.8 percent range had a median eight and a half publications and an H-index of seven.
"In a large cohort of NHLBI-funded cardiovascular R01 grants, we were unable to find a monotonic association between better percentile ranking and higher scientific impact as assessed by citation metrics," Lauer and his colleagues note.
Jalees Rehman at SciLogs points out that the study has a few limitations, namely that it relies on citations measures, which are not necessarily good indicators of scientific impact, and that it may not account for publications stemming from multiple grants well.
Still he adds that "[t]he results of this study are sobering because they remind us of how bad we are at predicting the future impact of research when we review grant applications."