By Julia Karow
Rapidly changing technology, complex sample preparation, unreliable instrumentation, insufficient data quality for some applications, and difficult and non-standardized data interpretation are among the challenges facing researchers trying to move next-generation sequencing into the diagnostic arena, according to several users of the technology who have been dealing with these issues.
The scientists, all users of next-gen sequencing platforms as well as lab automation equipment from PerkinElmer's Caliper Life Sciences, participated in a private roundtable discussion organized by PerkinElmer at the Advances in Genome Biology and Technology meeting in Marco Island, Fla., earlier this month.
Overall, there was strong agreement that today's next-generation sequencing systems do not fulfill the requirements of routine diagnostics yet.
"The research world and the diagnostic world are very different, and the companies that make the sequencers have not fully grasped that," said Toumy Guettouche, director of the oncogenomics core facility at the University of Miami's Sylvester Comprehensive Cancer Center. "Accuracy is the most important point in the diagnostic world; throughput is the most important aspect in the research world."
Guettouche, who is also in charge of genome technology assessment and implementation at the university's Hussman Institute for Human Genomics, runs a core facility and is also setting up next-gen sequencing in a CLIA lab. He currently has five Illumina HiSeqs and one Ion PGM.
Clinical applications of next-gen sequencing today are "more like research trying to fit the diagnostic helmet over the research realm and trying to make it work," said Guettouche, adding that he believes current technology is "at least two generations away" from the type of instrumentation that would be suitable for a typical diagnostic lab.
Shawn Levy, a faculty investigator at the HudsonAlpha Institute for Biotechnology in Huntsville, Ala., agreed. "Absolutely nothing is ready for clinical use," he said. While upcoming systems such as Illumina's HiSeq 2500 and Life Technologies' Ion Proton promise greater speed and throughput, which are good news for research, these advances are "in the completely wrong direction" from a clinical perspective, he said. "The reliability of the instruments and the quality of the data of the instruments is orders of magnitude [below] where it needs to be for accurate clinical use."
Levy's lab, equipped with eight Illumina HiSeqs, four GAIIs, one MiSeq, one Ion PGM, two 454 FLX, and one SOLiD 4, conducts both basic and translational research.
He noted that today's sequencers still frequently break down, reflecting the fact that they are cutting-edge technology, which is what researchers want to have access to. "But if there is a push toward diagnostics, there needs to be equal focus on continuing to make cutting-edge technology but also refining the technology to clinical robustness."
Some of this is already happening, he said. For example, Illumina's Genome Analyzer still required a separate cluster station, and loading the flow cells onto the machine was "very cumbersome." The cBot that came with the HiSeq was already "much more automated," and on the MiSeq, clustering happens on the sequencer and reagents come in cartridges. The next step for any type of sequencer, he said, will be an "accessory attachment" to load the DNA that performs the entire sample and library prep automatically.
Not Yet Ready for Diagnostics?
Data quality is also not sufficient for certain diagnostic applications, he said. For example, Illumina's sample indexing method currently has an error rate of 0.1 percent, meaning that some reads are assigned to the wrong sample. While that often does not matter, it becomes a problem for projects that look for things like low-frequency somatic mutations or circulating tumor cells. "Those applications are unattainable in a clinically relevant fashion because of these error rates," he said.
Several users agreed, though, that for many clinical applications, the single-read accuracy of the current sequencing platforms is sufficient because the consensus error rate after sequencing with several-fold redundancy is low. Small differences in accuracy that exist between today's platforms often do not make a big difference, Guettouche argued, and what counts is whether a platform is able to validate SNPs in a control sample.
"[People] argue whether the MiSeq is more accurate than the PGM, and they go back and forth. Well, do you get data that you can use from the PGM? Yes, you do. It might be not as accurate, or more accurate, than the MiSeq, but in the end, it's, 'Can you make your application work on the instrument?'" he said.
Also, it is still standard practice to validate clinically significant results from low-accuracy next-gen sequencing by capillary electrophoresis sequencing or other validation methods. Some users said, though, that next-gen sequencing sometimes trumps Sanger in accuracy, for example for calling heterozygous variants.
The complexity of sample prep, including target enrichment and library preparation, is another reason why next-gen sequencing is not yet ready for clinical use, users said. Many steps and enzymes are involved, and despite efforts to automate parts of the process, sample prep is often still "an art," according to Levy, with results varying from operator to operator. Differences in protocols, kits, and equipment — even the type of PCR thermocycler used — can also influence the results.
At the back end, the annotation and clinical interpretation of the sequence represents a major challenge, users agreed. "That will continue to be the major bottleneck for the use of whole-exome, whole-genome sequencing data in the clinic," said Levy.
One problem is that data analysis tools have not been standardized, even for research use. Using two different alignment tools on the same set of sequencing data, for example, can yield different and non-overlapping SNPs, Guettouche said.
Further, reimbursement for whole-genome or –exome tests will likely be hard to obtain, Levy said. While an in-depth interpretation is difficult and "no one is going to pay for it," interpreting only a subset of the data to arrive at a diagnostic conclusion is easier, but "if you are only looking at 10 percent, or even 2 percent, of the data, why would an insurance company pay for 100 percent?"
Putting NGS into Diagnostics Today
Those who are trying to put today's sequencing platforms to clinical use are struggling with the rapid technology advancements. "In a clinical setting, you have to pretty much have standardized workflows locked down, which is sometimes problematic because of how fast the field is advancing," said Darrell Dinwiddie, director of laboratory operations at the Center for Pediatric Genomic Medicine at Children's Mercy Hospital in Kansas City.
His team is developing a sequencing-based diagnostic test for approximately 600 childhood diseases (CSN 2/22/2012), using Illumina's target enrichment and HiSeq 2000 sequencer, that is performed in a CAP-certified lab. In addition, they perform exome sequencing for the diagnosis of "mystery cases."
During the validation phase of their test, for example, there were three major changes in reagents and protocols, requiring them to revalidate their samples. Every new piece of equipment and reagent lot needs to be validated, Dinwiddie said, which the lab is doing by sequencing the same Coriell control sample.
To keep up with improvements, he and his colleagues continue to evaluate new technologies that could make their test better, cheaper, and faster, but "you have to balance having to lock something down and keeping it going," he said.
Every variant of interest is currently also validated by capillary electrophoresis sequencing, though that is done not so much because the researchers don't trust the next-gen sequencing data — the mean coverage is about 1,000x — but because the test is currently performed under a research protocol and the results cannot be included in a medical record unless they have been validated. Longer term, he said, they are hoping to get away with no Sanger validation.
Dinwiddie questioned the assertion that the accuracy of the current next-gen sequencers is an issue. "I agree that we want to get it as accurate as possible," he said. "But the idea that every test that's ever done in the clinic is 100 percent accurate is completely wrong." The PSA test for prostate cancer, for example, has very poor sensitivity.
In the short term, clinical applications of next-gen sequencing will likely be focused and performed by CLIA labs that are experts in certain areas, according to Levy. Those labs will "optimize the sample prep, sequencing, and downstream analysis to validate their particular assay with a very high accuracy and robustness," he said. "But you won't be able to do that for whole-genome or whole-exome; it's too expensive."
Have topics you'd like to see covered in Clinical Sequencing News? Contact the editor at jkarow [at] genomeweb [.] com.