In an assessment using standardized RNA samples, sequencing platforms showed both intra-platform reliability and inter-platform agreement, according to findings presented by an Association of Biomedical Resource Facilities working group at the organization's annual meeting last week in Albuquerque, NM.
The results, which are from the first phase of the organization's long-running next-generation sequencing platform study that aims to develop methods and best practices, drew on RNA-seq experiments conducted on a set of standard RNA reference samples by some 20 core laboratories using a range of methods and platforms.
"[This study] is not a benchmark, but is to provide tools for self-evaluation and self-improvement," said George Grills, director of operations of the Biotechnology Resource Center at Cornell University and member of the ABRF executive board.
This initial phase of the ABRF-NGS study focused on developing RNA reference standards and evaluating various methods and tools, trying to find sources of noise within the generated data.
The two subsequent phases of the study, one of which is getting underway, will turn to reference standards for DNA sequencing and emerging sequencing technologies. The working group, led by Christopher Mason at Weill Cornell Medical College, noted that ABRF is also working in conjunction with the US National Institute of Standards and Technology to develop reference materials.
Grills said that the RNA-seq standards study is in press at Nature Biotechnology.
For the first part of the NGS study, more than 20 core lab facilities performed replicate RNA-seq experiment using a set of titrated reference RNA standards as well as a set of synthetic RNA spike-ins, Mason said.
Two standard RNA samples from the MicroArray Quality Control — MAQC-A and MAQC-B — were combined with spike-ins from the External RNA Controls Consortium (ERCC) to generate a set of six samples to be tested.
A variety of approaches, such as poly-A enrichment and ribo-depletion of degraded RNA samples, and size-specific fractionation were also included.
To generate their results, the cores used a range of NGS platforms, including the Illumina HiSeq and MiSeq, Ion PGM and Proton, the Roche 454 GS FLX+, and the Pacific Biosciences RS. Two RT-qPCR datasets were also derived as an orthogonal tool to evaluate the RNA-seq results.
The working group reported that the results from the study indicate high intra-platform consistency as well as inter-platform concordance, though it added that there were variable rates of efficiency and splice junction detection.
The inter-site median coefficient of variation ranged from 8 percent to 46 percent, though the researchers noted that all CVs were lower than they would be for randomly shuffled data.
Additionally, most junction sites were detected by three or more platforms, indicating concordance among the platforms. Novel junction sites were also uncovered, determined by their being detected by three or more platforms.
Long reads, like those generated by the PacBio platform, boosted splice junction detection efficiency, while higher read depth was needed to uncover rare isoforms, the researchers said. PacBio, 454, MiSeq, and Ion PGM could detect the two most common isoforms of the SRP9 genes, while the Ion Proton and the Illumina ribo-depleted or poly-A enriched preparations detected the third isoform.
"Every technology [has] its strengths and weaknesses," Mason said.
For instance, the researchers said that the MiSeq platform was good for rapid transcript quantification and detection while the Proton's strength was in good read depths and lengths for transcript quantification.
The researchers also examined different methods, finding, for instance, that ribo-depletion can both salvage degraded RNA samples and be compared to poly-A enriched fractions.
As the study offers a snapshot of the tools and methods currently available to the field, Cornell's Grills noted that the ABRF plans to develop a community resource so that the relevance of this work doesn't die out as the technology evolves.
The second phase of the ABRF-NGS study will be underway soon, said Don Baldwin, a working group member from Pathonomics. For it, ABRF will be working with the Genome in a Bottle consortium, organized by NIST to develop reference materials, methods, and data to assess human genome sequencing performance.
"We're very excited about this development," Grills said.
This second phase of the ABRF study will examine genomic DNA from an Ashkenazi Jewish family trio from the Personal Genome Project on a number of platforms — including the Illumina HiSeq X Ten and Moleculo tools — and a range of protocols, including not only vendor-specified ones, but also Nextera, NuGen, and other kits, to examine the performance of these approaches.
The second phase will also examine FFPE samples as well as bacterial genome samples with a range of GC content. Those bacteria, chosen by the US Food and Drug Administration, include Staphylococcus aureus and Pseudomonas aeruginosa, among others.
The goal, the University of Vermont's Scott Tighe said, is to assess the sequencing accuracy, coverage, and limits of sensitivity.
Tighe added the group anticipates finishing the design of the second phase of the study by the middle of June, collecting the data by May 2016, and finishing the paper a year after that.
The third phase, which will assess emerging technologies such as the GnuBio, BioNano, and Qiagen platforms, among others, is scheduled to begin after the second phase is under way.