Peer-Review Uh-Ohs

Sometimes researchers make mistakes. But how often are researchers citing papers that have nothing to do with the point of their own research? More often than you'd think, says NeuroDojo's Zen Faulkes. He cites a new paper by Peter Todd et al. in the Marine Ecology Progress Series that says that as many as one in four citations in marine biology papers is "inappropriate." But the picture isn't as bleak as Todd would make it out to be, Faulkes says. "The 'one in four' comes from their claim that 75 percent of references support the claim in the paper making them. The remaining 25 percent that they deem 'inappropriate,' however, fall into several categories, and not all of those categories are equally problematic," he says. About 10 percent is ambiguous, and about 7.6 percent of what Todd and company deemed wrong are most likely cases of people citing review articles, for which there are positive reasons, according to Faulkes. Impact factor is also unaffected, he adds, because there is no evidence of bias in any of these mistaken citations, and Todd's paper seems to suggest that the errors are random. Bottom line for Faulkes: "Journal editors and reviewers should be the gatekeepers here." Reviewers should know about the relevant literature so they can catch mistakes before they're printed. Maybe journals should even hire some fact-checkers, he suggests. "That [Todd's] article shows how easy it is to go and fact-check these things shows, more than anything else, that the citation system works. We just don't take enough advantage of the opportunities."