NEW YORK (GenomeWeb) — Researchers interested in microRNA expression profiling today face a bevy of options including microarrays, quantitative PCR assays, and next-generation sequencing applications, but it is not always obvious which platform is the most suitable for a given study.
To make those choices clearer, a team at Ghent University in Belgium recently led a microRNA quality control study, dubbed the miRQC, in which they evaluated nearly a dozen different platforms sold by some of the industry's biggest vendors on the basis of reproducibility, sensitivity, accuracy, specificity, and concordance of differential expression.
The results of the quality control study were published online recently in Nature Methods.
While the authors reported that qPCR assays maintained the highest sensitivity and accuracy, the picture was not entirely bleak for hybridization-based platforms such as arrays, some of which demonstrated the highest reproducibility of those evaluated thanks to the presence of replicate probes.
"Hybridization-based platforms in general are less sensitive compared to next-generation sequencing and qPCR," said Pieter Mestdagh, a researcher at Ghent University and lead author on the paper. "On the other hand, array platforms are typically very reproducible and this reproducibility allows you to better quantify expression differences," he told BioArray News.
Mestdagh and colleagues decided to carry out the miRQC study because of the laboratory's history of running miRNA studies, mainly in oncology, and the dizzying number of different offerings for miRNA expression profiling on the market.
"It's amazing how many providers have jumped into this market and I can imagine that if you are new to the market, it can be difficult to decide on what platform to use for your study," said Mestdagh. "Frequently people ask us what platform we use and we came to the conclusion that there had never been an in depth platform comparison," he said.
To carry out such an evaluation, the miRQC team contacted most of the major vendors in the market, of which 10 agreed to take part, including Affymetrix, Agilent Technologies, Exiqon, Illumina, Life Technologies, NanoString Technologies, Qiagen, Quanta Biosciences, Toray Industries, and WaferGen Biosystems.
"Overall, I think we covered 95 percent of the miRNA market," said Mestdagh.
Each participating firm received 20 anonymized RNA samples and was asked to profile at least 384 miRNAs according to their standard procedures. The samples included human universal reference RNA, human brain RNA and titrations, human serum samples, and synthetic spikes from miRNA family members with varying homology.
According to the paper, one vendor, Toray, a Japanese firm that entered the European market with arrays for gene and miRNA expression profiling three years ago, withdrew its data and so its data set was excluded from the study.
And though Denmark's Exiqon offers a line of miRNA microarrays, and was among the first companies to do so when it debuted the chips in 2005, it opted to participate in the comparison with its qPCR assays, which have grown in popularity among customers in recent years.
In total, the Ghent University-led team assessed six different qPCR offerings provided by Exiqon, Life Technologies, Qiagen, Quanta Biosciences, and WaferGen; three different hybridization platforms offered by Agilent, Affymetrix, and NanoString; and two NGS applications run on Illumina and Ion Torrent sequencers.
To develop metrics for assessing the different platforms, the authors drew in part on the Microarray Quality Control project, an effort led by the US Food and Drug Administration that compared the reproducibility and cross-platform comparability of eight different microarray platforms.
"They developed quite a lot of interesting ways of looking at expression data and designing samples in such a way that you can evaluate platform performance," said Mestdagh. "We further developed a set of novel and robust performance metrics." Among the metrics used included titration response, sensitivity, specificity, and reproducibility.
According to the authors, when looking at platform correlation based on all metrics, there were "no obvious clustering of technologies," meaning that the results for the different qPCR, hybridization, and sequencing platforms were varied enough so that no one technology performed the best across all of the different metrics.
In terms of both sensitivity and accuracy, qPCR platforms performed better than hybridization and sequencing platforms, especially when it came down to low-input RNA samples. At the same time, the authors excluded Life Technologies’ OpenArray and WaferGen's SmartChip platforms from these conclusions. The authors also noted "poor titration response and reproducibility for several qPCR platforms," which, according to the authors, "implied that these qPCR platforms should not be used to quantify small changes in expression in small size sample cohorts."
The results "demonstrated that platforms based on the same technology can have very different performance," they wrote. "This is most obvious for reproducibility and specificity among qPCR platforms."
At the same time, WaferGen's PCR assays, Illumina next-generation sequencing application, and Agilent's arrays all showed "exceptional reproducibility," according to the authors. Mestdagh attributed this performance in part to the use of replicate probes by Agilent and WaferGen. He noted that Agilent was the only company with a hybridization-based offering that agreed to submit data highlighting the performance of its arrays to detect miRNAs in serum. According to Mestdagh, such attributes could make such platforms optimal for looking at samples distinguished by very small expression changes.
"Because of their higher reproducibility, hybridization platforms will better allow you to pick up those small differences," said Mestdagh.
The authors found that sequencing platforms are sensitive when RNA is not limiting, but lose sensitivity for low-input-amount RNA samples, such as serum samples. However, Mestdagh noted that "there is evidence that modified procedures exist that will allow you to do miRNA expression profiling in serum," adding that the "technology is evolving."
Another finding the miRQC encountered was a relatively modest overlap between platforms, regardless of the underlying technology. According to the authors, there was an average validation rate for differentially expressed miRNAs of only 55 percent between any two platform combinations, leading them to "strongly advise that screening studies are followed by targeted validation using an alternative platform or technology," results that may encourage researchers to use multiple technologies going forward.
"When comparing any two platforms, the overlap was only about 50 percent, which is low," said Metsdagh. "Depending on the technology you use, you might get different results, so we are advising people to validate their results using a different technology."