NEW YORK (GenomeWeb) ― Though reverse transcription of RNA to cDNA is an essential first step for a growing number of genomics applications, researchers have long known of problems associated with RT and its effects on study results. Now, a new study has examined some of those issues and has provided a workflow and guidelines for researchers publishing RT-based data.
The study, published in Clinical Chemistry late last month, was undertaken by some of the authors of the Minimum Information for Publication of Quantitative Real-Time PCR Experiments, or MIQE, guidelines, which debuted in 2009. It compared measures of different mRNA targets using six commercially available RT enzymes and varying sample concentrations of differing qualities. Results showed the variability was "sufficiently large to call into question the validity of many published data that rely on quantification of cDNA."
Author Stephen Bustin, a professor of medicine at the UK's Angela Ruskin University, is a bit of a crusader for data standards in RT-qPCR. In a conversation this week with PCR Insider, he explained the motivation for his mission, as well as the recent "disconcerting" results.
"We've known for a long time there's a problem [with qPCR results reporting]," Bustin said. "I think what really brought it to a head was my involvement with the measles, mumps, and rubella autism story, where I saw for myself what impact poor research has on people. ... Sometimes life and death does depend on what we publish, and that's what motivated me at the time to [work on] the MIQE guidelines; I saw first hand how poor research, poor protocols, and poor science affects people."
The new study cites a growing body of publications on reproducibility of results, particularly with respect to molecular methods.
Currently, there is a disconnect between the numbers of RNA-based biomarkers being reported and how useful they are, Bustin said. He referred to a recent paper in The Lancet titled "Biomedical Research: Increasing Value, Reducing Waste," which noted that there were more than 1,500 reports about cancer prognostic markers in 2005, few of which have entered routine clinical practice nearly a decade later.
A group of more than 80 scientists also authored a paper last year in Nature Methods which looked at nearly 1,700 publications that used qPCR and found "a real problem" in the quality of data reporting, Bustin said.
This is similar to a well-known 2012 paper by Begley and Ellis, which described an inability to reproduce peer-reviewed preclinical oncology data from 47 out of 53 undisclosed published cancer studies, Bustin noted.
Bustin's group also published a study earlier this year in Molecular Oncology which scrutinized 179 published qPCR-dependent RNA biomarker assays for colorectal cancer prognosis.
"What we showed is we can't really rely on these results, and if we extrapolate ... most studies that use qPCR cannot be reliably assessed," said Bustin. That study also concluded it is possible that, "at worst, pervasive incorrect normalization is resulting in the wholesale publication of incorrect conclusions."
The recent Lancet paper concluded that this so-called reproducibility crisis equates to about 85 percent of research investment being wasted, according to Bustin.
"It is frightening," he said.
Reverse transcription is central to many techniques, such as qPCR, sequencing, arrays, and probe generation. In turn, these are used in a wide range of different applications. However, problems with the RT step are not unknown to researchers. Bustin's group has been describing the technical limitations of this step since 1998, particularly the variable efficiency. And a critical paper in 2004 demonstrated "wildly different" cDNA yields, depending on target, concentration, and enzyme, Bustin added.
The Clinical Chemistry study now documents that RT results are dependent on the transcriptase, target, assay, sample, and concentration, and that "in general, the sort of fold changes you can get without really trying too hard are two- to three-fold; that would be a typical fold change ... with a good reverse transcriptase, doing everything properly," Bustin said. He also noted that low concentration of sample can increase variability up to nine-fold.
Noting that two- to eight-fold changes are frequently published and touted as evidence of biologically relevant differences between control and experimental groups, "People need to be far more careful in how they interpret their results," Bustin said.
Although some scientists have pointed out the risks of the current emphasis on reproducing experiments, the reproducibility crisis in biomedical research is currently being tackled by journals as well as advocates like Bustin. Thirty major journals recently agreed with the National Institutes of Health on a common set of principles and guidelines in reporting preclinical research.
Bustin sees a number of ways forward for RT-qPCR. He suggests multiple RT steps to get an idea of variability prior to the qPCR step. The "much-maligned" single tube RT-qPCR can also yield less variable results, Bustin said.
For biomarker studies, "You have to make sure the integrity and purity of each individual RNA sample is equivalent," Bustin said. PCR efficiency, reference genes, and data analysis steps are important. Publishing a detailed protocol of methods and rationale can also help improve confidence in results.
A good reverse transcriptase, and attention to buffer, is also vital. Of the six reverse transcriptases evaluated in the Clinical Chemistry study, the ReadyScript from Sigma-Aldrich and iScript from Bio-Rad Laboratories "were consistently good enzymes," Bustin said. He suspects the enzymes may not be all that different structurally, but rather some property of the buffers they are supplied in makes them function more reliably.
"What we find is that we always get good results with those two enzymes," he said. Notably, although the enzymes tested were "free samples" from the companies, there was no commercial collaboration on this study.
In addition to rigorous proofs of the consistency of any given assay, digital RT-qPCR seems to give less variable results, and Bustin said his group will soon publish data on this subject. With dPCR, "We're beginning to get to a point where we can perhaps be more precise," he said. Still, this does not help with the RT step, and "there seems to be no other way of doing that step at the moment that anyone has invented," Bustin noted.
For now, when asked for an example of one "good" study that has met the minimum guidelines for RT-qPCR, Bustin said, "We [recently] looked at 400 papers ... there wasn't a single one. It is very rare to find anything that makes any sense, unfortunately."