A team of NASA researchers published a paper in Science in December 2010 in which they described a microbe called GFAJ-1 that they said could substitute arsenic for phosphorus in a percentage of its DNA backbone. But University of British Columbia microbiologist Rosie Redfield begged to differ and did so publicly on her blog, RRResearch. After careful examination, Redfield and many of her colleagues in the microbiology community were alarmed by the methods used in the paper, and found evidence that contamination may have occurred.
Redfield blogged her criticisms of the research and her incredulity that the paper had made it through the peer-review process. In a proof-of-concept demonstration of her belief in open science, she documented her unsuccessful attempts to replicate the paper's findings. In January, Redfield submitted the results of her research to the preprint server arXiv. Genome Technology's Matthew Dublin spoke with Redfield about the study and open science. What follows is an excerpt of their conversation, edited for space.
Genome Technology: What was your initial reaction to the Science arsenic paper?
Rosie Redfield: All of the scientific consensus was that the conclusions in their paper were wrong. They were contrary to everything we felt we already knew, and the data that supported them was very weak and compromised by contamination. We set out to do what we felt was the minimum experiment that would destroy the most striking conclusions of the paper, which was to grow the cells with and without added phosphate, and with and without arsenate. Then we needed to show that the cells didn't need arsenate to grow, and to purify the DNA properly — much more properly than the original authors had — and to show that there was still arsenate in the DNA.
We didn't want to fall into the trap of 'Well, we know what we expect our experiments to show, so we're only going to do the experiment that will gives us the answer we expect' — we wanted to do well-controlled, careful experiments. So we grew cells in four different conditions [and] purified the DNA, analyzed it, and the answers were exactly as we expected: The cells grow fine without it, the bacteria don't need arsenic to grow at all, and when you give them lots of arsenic and hardly any phosphorus, they don't put detectable levels of arsenic into their DNA.
GT: Why refute the paper so publicly?
RR: It was this sense of outrage. As one of the thousands and thousands of scientists who would love to get a paper published into the journal Science — which is very competitive — and who are all struggling to do careful, meticulous work in the hope that we will do something important enough to get published in Science, I was really offended by how terrible this paper was and that Science published this really deeply flawed paper, which fell far below the standards of all the scientists in the country. That was the personal feeling about it.
I thought it would be good if I did it because I could use it as an example of doing science in the open, posting my success and failures on my blog. I wanted to push the idea that it's possible to do science in the open and it can be beneficial to do science in the open. For most scientists, that's a very startling idea.
GT: What does it say about the peer review process that this paper got published?
RR: I'm a big fan of peer review, but for this paper it's just difficult to believe what we were told — that the Science reviewers found only minor flaws. This was inconsistent with the severe flaws that readers found in the published paper. I hypothesize that the Science reviewers might have missed the errors because they had specialist analytic-method expertise but not general microbiology or molecular biology expertise.
There is also another level in that we cannot forget that [lead author Felisa] Wolfe-Simon is left high and dry defending the work, but there are 12 other authors who have not taken any real responsibility for the quality of the work. The problem may be that a number of these authors were part of the paper because they brought a particular technical expertise to the problem and that they feel that being an author is how they were paid for their technical expertise, so they may feel that they have no responsibility for the rest of the quality of the paper.
GT: Why do you think this paper was published without being reviewed by microbiology experts?
RR: The problem is not that the reviewers are stupid — it's that it was sent to reviewers with inappropriate specialized expertise. But in addition to none of the other authors coming forward and defend the manuscript, none of the reviewers have come forward to say that they reviewed this paper and thought it was good because of this and this, and 'I didn't know enough about X to worry about this other problem.' It's like this conspiracy silence.
GT: Why is blogging one's scientific research important?
RR: It's not just that you can communicate your points to one other scientist who will pass it on to somebody else — now you can put your opinions where they are publicly available so that anybody who is interested can see that you said X on this topic. Blogs and Twitter are changing the dynamic of communication and giving you the ability not just to do one-to-one communication, but to broadcast your ideas. ... You can really amplify the power of good thinking.
GT: What does the controversy surrounding this paper and your blogging efforts mean for open science?
RR: To use an old term, it's a consciousness raising for open science. It's providing an example in a situation where a substantial number of people will notice it, so I'm really happy that the coverage of this will — just in part — focus on the open science side of it. Scientists have lots of reasons for communicating or not communicating about what they do, and all that this will do is show people that they could do this if it suits their circumstances or interests.
GT: What do you hope is the reaction to your paper?
RR: I hope researchers will see this whole fiasco as a warning of how terrifyingly easy it is to delude ourselves that our experimental results support our beloved theories, as well as the importance of opening our work to criticism at every step of the way.