Some 157 open-access publishers accepted a fake, fatally flawed manuscript from Science's John Bohannon, while 98 rejected it. Bohannon did not send his manuscript to traditional journals.
To bring about this sting, Bohannon created what he dubs a "scientific version of Mad Libs." The manuscripts he sent off followed the form of "[m]olecule X from lichen species Y inhibits the growth of cancer cell Z" in which he substituted different molecules, cell lines, and lichens for those variables. Otherwise, he says, the papers were identical and contained numerous errors, including data plots that showed the opposite of what the paper claimed.
He then sent 304 of those faked manuscripts to journals he culled from both the Directory of Open-Access Journals and a list of predatory open-access journals kept by Jeffrey Beall, a library scientist at the University of Colorado, Denver. Bonhammon focused on gold-model journals where authors pay publication fees.
"Acceptance was the norm, not the exception," Bonhammon writes, noting that the faked papers were accepted by journals that fell under the umbrella of large academic publishers, prestigious institutions, and academic societies.
Some open-access journals, including PLOS One, did reject the manuscripts.
"Bohannon's analysis … demonstrates an appalling lack of peer review and quality control at the journals he spoofed. But it's important to note, given the heated and endless debates between open access advocates and traditional publishers, that there was no control group," says Ivan Oransky at Retraction Watch.
Indeed, Martin Eve from the University of Lincoln in the UK argues at the Conversation that what Bohannon uncovered deals more with the flaws of peer review than with open-access journals. "It’s about a flawed system of trusting journals and the inherent problems in peer review, but he targets only open access here," Eve says.
In an interview with NPR, Bohannon acknowledges this shortcoming, saying that since he didn't do the control of submitting fake papers to subscription journals and, as such, cannot conclude that open-access journals are worse than traditional ones. He adds, though, that online publishing makes it easier for poor-quality journals to exist.
After journals accepted any faked manuscript for publication, Bohannon withdrew it from publication.