William Gerin says that writing competitive grant proposals is as much about mastering the game of peer review as it is about selling the science.
"We tend to think that as long as we've got the really good science, the great idea, the cutting-edge methodology, then we should get funded. And indeed, you should, but it's not like that," says Gerin, a professor of behavioral health at Pennsylvania State University. "Long before I ever start writing down my cherished idea, I've got to be aware of how the system works. Who is it that's reviewing my grants? To which study section is it likely to go? Who sits on that study section? These are all elements of the game."
The National Institutes of Health's Center for Scientific Review invites grant applicants to suggest a specific study section assignment in the cover letter that accompanies their applications. Toni Scarpa, who directs the CSR, says that "about 60 percent of applicants who submitted applications for new investigator-initiated research in 2010 suggested a specific study section assignment for their applications." For the most part, CSR honors these suggestions. Only when "we determine that a suggested study section would be inappropriate or unfeasible [do] we assign the application to a more appropriate study section and inform the applicant," he says.
In the interest of transparency, CSR has published descriptions of its study sections, as well as their rosters, online. Of all the center's resources, Scarpa says that these two "have become some of the most frequently used pages on our Web site." According to Scarpa, CSR's study section description pages were viewed nearly 1.7 million times in 2010, and the link to the section rosters is the 11th most-accessed page overall. Because of their popularity, CSR redesigned its homepage last year to highlight those resources.
Still, to Gerin's mind, most researchers don't make use of the publicly available resources, like reviewer rosters, when writing grants. "A lot of time when I suggest it to people, it's like, 'Oh, what an obviously good idea!' For some reason, it just doesn't occur to them," he says. "It doesn't seem to have occurred to them that that is the kind of preparation that goes into their proposal just as much as the science does."
Review the reviewers
Though suggesting a study section assignment does not guarantee an application's placement there, researchers can still put the reviewer rosters to use. CSR frequently updates its index of standing study section members — who review most investigator-initiated grant applications — as well as its listings of those who serve on special emphasis panels to evaluate small business innovation research, technology transfer research, and fellowship proposals.
"Applicants would do best to focus on our study section descriptions to understand the scope of the science covered by the study sections," Scarpa says. "Examining recent study section rosters can aid in this understanding," he adds, though he notes that these "can change significantly from round to round since we recruit many, if not all, reviewers for a meeting based on the specific scientific content of the applications to be reviewed." CSR typically makes revised rosters available within 30 days of scheduled meetings.
'Try to identify one'
While one roster may contain 20 to 40 members, CSR typically assigns at least one reviewer from outside an applicant's discipline to include a variety of perspectives. Researchers ought to familiarize themselves with anyone who might have a say in their application's fate, Gerin says. For any single study section, he says, "you're going to get people with different kinds of expertise."
For example, for an unsolicited R01 genome-wide association study proposal, geneticists may be assigned to review and the applicant ought to "be aware [that] the kind of evidence they want to see is going to be different than the kind of evidence an epidemiologist or public health [researcher] is going to want to see. You should be sensitive to that; you're talking to these people and they've got to be able to hear what you're saying," Gerin says.
Having scanned the range of expertise among potential reviewers, the applicant should then "try to identify one person who might be on your study section," he says. Young investigators, who may not yet have the experience needed to prune probable picks from the scores of reviewers on their own, ought to ask their senior colleagues for advice as to who a likely reviewer might be.
After homing in on one person, the applicant should then become familiar with that individual's work. "If they're somebody whose work overlaps yours and you didn't cite it, you look stupid," Gerin says. "Read some of their papers. It's not just a matter of 'Let's work their names into the citation list,'" as reviewers are quick to notice that, he adds. If the person's published results contradict those in an applicant's proposal, Gerin says it's best to acknowledge the disparity up front. "Cop to it. You say: 'Not everybody has found this, [or] supported it,' or 'There is some controversy,' and so on," he says. At the very least, avoiding the discrepancy reflects poorly on a researcher's knowledge of the literature. At worst, the reviewer could feel insulted.
Still, a researcher should not attempt to mold a proposal to a potential reviewer's liking. "Write your passion; do the research you really want to do. Don't tailor it to the mechanisms that have more money … or to the preferences and likes of a potential individual study section person because it comes through," Gerin says. "The research that gets funded is the research that people believe in. That really comes through in the application. My advice is: don't try to out-game the system by forcing your research into a niche to make it ... palatable to particular reviewers or to take advantage of a grant mechanism that has a slightly higher success rate."
While he believes peer review is indeed a game, Gerin says there's no profitable way to cheat it. "It's not about gaming the system, it's about understanding the need of the reviewers to hear the things that reassure them they've got a good application," he says. In order to prepare an airtight grant proposal, "you really have to get into the minds of the reviewers."
How often you scan the roster of the study section that is most likely to review your NIH grant proposal?
36% Every time. It's an integral part of my grant-writing routine.
12% Sometimes. I've scanned the list before, but the lineup only changes every few years.
5% Rarely. Only if my proposal seems likely to be assigned to a study section new to me.
10% Never. Who has that kind of time on grant deadline?
35% Never. You can look up study section rosters?