Guidelines proposed in early 2009 to help standardize how qPCR results are reported are "slowly filtering through" the research community, but much work still needs to be done to improve the quality of published qPCR studies, according to one of the authors of the standard.
The Minimum Information for Publication of Quantitative Real-Time PCR Experiments, or MIQE, guidelines, published in Clinical Chemistry in February, outline the essential information that authors should provide to serve as a benchmark for assessing the quality of qPCR assays reported in published studies.
The guidelines include a "checklist" intended to help authors ensure that their qPCR-based experiments can be reproduced by others, and steps journal editors and reviewers can follow to help them assess the quality and validity of submitted research.
The document was intended as a "first step" toward standardizing research results in the field, Stephen Bustin, professor of molecular science at the Barts and the London School of Medicine and Dentistry at Queen Mary University of London and first author on the MIQE paper, told PCR Insider. Its "main aim" was to "really just put forward a discussion document that deals mainly with expression analysis for messenger RNA."
Indeed, the paper doesn't touch on subjects of interest to the broader PCR community, such as how to analyze microRNAs and SNPs, use high-resolution melt analysis and multiplexing, or consider the entire realm of diagnostic applications for qPCR — topics that the MIQE authors intend to address in future versions of the guidelines.
Bustin said that he and his co-authors recognize that adoption of the MIQE guidelines — even just for gene-expression research — "is going to require a sea change in people's attitudes."
He said that as he and his colleagues have discussed MIQE at conferences over the last year, they've found that most researchers are "enthusiastic" but harbor some resistance due to the perception that it will "add significantly to the workload" when publishing a paper.
Bustin said those doubts often dissipate "once we explain to them that, really, in the course of designing a set of experiments and then carrying them out, most of the information that is required [for MIQE] actually should be accumulated and collated anyway" as part of sound scientific practice.
"No one is being very negative," Bustin said. "Most people understand why we're trying to do this, and the problem is that if you have been doing something in a certain way, then it’s difficult to accept or to acknowledge that you might have been wrong."
He added that it appears that the MIQE guidelines are "actually very slowly filtering through to people."
Nevertheless, he noted, the guidelines appear to have made little or no impact on the quality of the published literature over the last year — largely because most journals have not enforced them.
"I do quite regular surveys of the papers that are being published, and I see no real evidence of any great improvement in the information that is being provided," he said. "The quality of papers that are being published, using in particular RT-qPCR, is still pretty appalling."
In some cases, he noted, "it’s actually worse than it used to be; there’s actually less and less information — even online. Sometimes the information is relegated to two or three lines in the underlying methods section." Not only does this lack of detail make it impossible to reproduce the experiment, he said, but it could conceal faulty methods that might have generated inaccurate results.
"I think the main problem we face is not so much that researchers or companies don't want to help implement and spread the guidelines, but it’s the fact that the journals and journal editors seem to either be reluctant or unaware of the need to do something about the standard of work that people are producing," he said.
Stepping Up to the MIQE
Bustin said that he and his colleagues have been "encouraged" by the level of support from instrument and reagent suppliers. He singled out Bio-Rad, Thermo Fisher Scientific, and Life Technologies' Applied Biosystems for being "good examples of companies that have take a real interest in implementing the MIQE nomenclature and also training their staff and sales teams" to comply with the guidelines.
He noted that Thermo is training its sales and technical staff "to really pass on the MIQE methods," while Bio-Rad "has had a series of European meetings this year where myself and others spoke, and the main purpose of these meetings was to let people know about MIIQE."
[ pagebreak ]
In October, Bio-Rad announced that it was providing Biogazelle's qbasePLUS qPCR data-analysis software along with its CFX96 and CFX384 real-time PCR detection systems, in part because it "conforms" to MIQE.
Steve Kulisch, marketing manager for Bio-Rad's gene-expression division, told PCR Insider that the company is "doing whatever we can to make sure that customers who want to be compliant with MIQE can be."
He noted, however, that the company doesn't expect that "every single one of our customers" will choose to be compliant. "We think that the guidelines laid out in MIQE make a lot of sense, but we understand for a lot of practical reasons that there are researchers who may not be able to adhere to it."
Thermo has also taken steps to ensure that its products are in line with the MIQE guidelines. Its Solaris qPCR assays, for example, include the sequence information for all probes and primers, which "improves publication quality and follows MIQE recommended guidelines," according to the product literature.
This feature is critical for pre-designed commercial assays, Bustin noted, since many vendors do not provide the sequence information for primers and probes. The MIQE checklist deems primer sequences to be an "essential" requirement, while probe sequences are "highly desirable and strongly encouraged." However, they can't be considered essential because "not all vendors of commercial predesigned assays provide this information."
The MIQE checklist "discourages" the use of these proprietary commercial assays.
Bustin said that vendor support is key for the guidelines to be adopted. "I think if other companies start supplying [MIQE-compliant] analysis programs with their instruments, and if they train their staff so that they are familiar and supportive of the MIQE guidelines, then it would start filtering out among people who are using real-time PCR."
Yet despite these signs of support for MIQE in the community, the effort has encountered some hurdles — most notably a reluctance by scientific journals to enforce the guidelines.
Journals, understandably, are primarily concerned with the scientific findings of a study rather than the underlying technology that produced those results. However, it appears that some publications fail to recognize that if the underlying methodology is faulty, "then the whole edifice of the paper crumbles," Bustin noted.
There have been several high-profile cases where misinterpreted PCR results led to incorrect conclusions. In one case, Science in 2007 was forced to retract its 2005 "breakthrough of the year" — a study that described the migration of mRNA to initiate flowering — because the authors incorrectly analyzed the real-time RT-PCR data.
"When all the primary real-time RT-PCR data are subjected to correct statistical analysis, most of the reported significant differences between time points disappear," the retraction explains.
In addition, Bustin noted that the link between the measles, mumps, and rubella vaccine and autism, first reported in 1998 and later discredited in a number of follow-on studies, "was buttressed by real-time PCR data that turned out to be fictitious and inappropriately analyzed."
In most basic research studies, inaccurate PCR results "won't make an awful lot of difference in the great scheme of things, but when you’re dealing with people's lives based on inappropriately carried out PCR experiments, then that’s a different matter," he noted.
[ pagebreak ]
The support of journals is crucial because, "obviously, if people can publish their data without having to supply the information to reviewers or readers that allows them to evaluate whatever technology is used in those papers, then people aren't going to bother doing it," said Bustin. "Why make more work for yourself if you don't have to?"
The problem, he noted, is that "publications like Nature seem to think that it's an established technique and there’s no problem with it." PCR is "perceived to be simple and straightforward. But obviously anyone who has any detailed understanding of the technology knows that it’s none of these things."
Nature has adopted similar standards in the past. For example, in 2002 the journal endorsed the MIAME (Minimum Information About a Microarray Experiment) guidelines, noting that all submissions to all Nature journals based on microarray experiments must include CDs with "necessary information compliant with the MIAME standard."
Nature officials could not be reached for comment for this article.
Bustin said that he and his colleagues have written letters to several journal editors in the hopes of convincing them to begin enforcing the standard, but they have seen little success to date.
In the meantime, "I’m on the board of a number of journals, and a number of my colleagues are, so [for] the journals that we have any influence over, we do suggest that people look at the guidelines and submit the important information if it isn’t provided already."
He acknowledged that getting researchers to comply with the standards will take time.
"Obviously you can't expect people to accept a radical departure from one day to the next, so it’s going to be an incremental increase in getting people to publish small steps first."
As an example, he said, "if you’re dealing with a gene, you publish the accession number, you publish details of how you did your reverse transcription. And once you start doing that, I think you’re already giving more information than a lot of papers have provided in the past."
He added, however, that the best way to enforce adoption of the guidelines would require "a concerted effort by editors to insist on the introduction of at least a core set of rules and guidelines for publication."
One possibility that Bustin and his colleagues are considering is revisiting the original checklist to "come out with a shortened version that highlights the issues that are really important."
He said that he and the other MIQE authors will be discussing over the next few months "how we're going to go forward with either producing an abridged and revised version of what we've already got, and/or incorporating some of the issues that have been perhaps left at the periphery of the first version of the MIQE guidelines."
One area under consideration is diagnostic assays, which "is a slightly different issue because in general I think people are more rigorous in their diagnostic assays," he said.
Diagnostic assays "are probably more standardized already, they’re more reliable, and they are certainly less controversial, I think, than in research publications where everyone does their own thing," said Bustin.