By Monica Heger
As targeted sequencing panels are increasingly being used in the clinical setting, labs offering these assays as laboratory-developed tests must find their way through an uncertain regulatory landscape, particularly if they are using kits or reagents that are marketed for research use only.
While a number of vendors are now offering sequencing reagent kits targeted at specific disease areas, US Food and Drug Administration guidelines require them to label these kits as RUO products if they have not sought FDA approval for them. And while clinical labs have historically used RUO reagents as the basis for lab-developed tests, the agency recently began cracking down on this practice, issuing a draft guidance document last June that warned labs and vendors that such products should not be used for clinical applications (PGx Reporter 6/8/2011).
So far, the US Food and Drug Administration has not issued any regulations specific to the use of RUO products for next-gen sequencing, but it has said that it plans to more closely monitor sequencing-based LDTs.
In the absence of clear guidelines, clinical labs are left on their own when it comes to validating tests based on these products.
Ephrem Chin, manager of Baylor College of Medicine's mitochondrial diagnostic laboratory, told Clinical Sequencing News that in the current regulatory environment, labs bear all responsibility for defining what their assays can do. They have to make sure the test is reproducible, establish quantitative measurements for sensitivity and specificity, and identify its limits of detection.
"Basically, a lab's charge, if they want to develop an LDT, is to establish a value for all these criteria and make it available to any inspection team or regulatory agency, if requested," said Chin. This internal validation requires a great deal of in-house expertise and can take up to a year in some cases.
Ideally, Chin said that kit manufacturers would take their assays through FDA 510(k) clearance, but this will require FDA-cleared NGS platforms. While the field is headed in this direction — with both Illumina and Life Technologies pledging to take their MiSeq and Ion Torrent PGM machines, respectively, through the FDA approval process — labs looking to run NGS-based tests now must take a do-it-yourself approach to validation.
And while FDA involvement may be welcome when it comes to the availability of 510(k)-approved kits, there is a great deal of uncertainty about the agency's intentions in the meantime with regard to regulating research-use-only reagents used as lab-developed tests.
"The practice of taking in RUO products and building a clinical test is so commonplace now that if the FDA decides to regulate this field and reagents are not available to laboratories to do this specialized testing, it basically will close down a whole segment of industry," Chin said.
Self-Validation
Chin said that most clinical labs offering sequencing-based assays follow guidelines published by the College of American Pathologists in 2009 for validating molecular tests in a clinical lab.
As an example of the internal validation process, he described Baylor's 16-gene test for glycogen storage disorders, which uses NimbleGen's SeqCap enrichment kit, marketed for research use only.
To use that kit for a clinical test, Chin said the team first had to establish its accuracy in being able to capture all of the genes of interest, as well as establish any off-target genetic material it captured.
Then, he said, the kit had to be tested within the lab's next-gen sequencing pipeline. The team establishes whether the number of reads generated is enough to detect the disease-causing mutations, including single polymorphic changes and deletions and insertions.
One of the biggest challenges, he said, is determining a threshold for the size of deletions or insertions an assay can detect.
"Each lab has to establish its own confidence level for detecting indels," Chin said. For instance, he said, when he was at Emory Genetics Laboratory, the team determined that it could identify indels up to 55 base pairs.
As part of the validation process, the assay is run on wildtype samples to make sure the targets are represented accurately and specifically, and that normal polymorphisms are detected. It will also run samples that have already been diagnosed to see if the assay picks up everything that has been previously reported.
Next, the lab runs samples that had been previously diagnosed as negative but still have clinical presentations indicative of a glycogen storage disorder, to see if the new assay can pick up any more mutations.
For instance, Baylor's 16-gene assay identified causative mutations in 70 percent of the cases deemed negative by single-gene tests, Chin said.
Finally, if the lab has multiple instruments and multiple people running those instruments, it has to ensure that each instrument and each person gets the same results. Identical samples will be run on different machines and by different people to demonstrate that the data is consistent and accurate.
After the initial large validation, smaller validations have to be done each time a new batch of reagents is brought into the lab, Chin said. For that, he said, labs should "run a sample that we've run previously, to show that [the reagents] work."
These same steps have to be followed for labs launching clinical tests on kits pre-designed for specific diseases, he said, such as RainDance's autism panel or Ion's AmpliSeq Cancer panel.
A research-use-only kit can't be taken straight out of the box and run as a clinical test, he said.
For instance, Greenwood Genetics Center currently offers an autism test based on RainDance's ASD-seq panel, which was developed for research use only.
Julie Jones, director of the molecular diagnostic laboratory at Greenwood, said that the lab did "in-house validations and quality controls" of the panel, which enabled them to launch it as an LDT.
In fact, through the validation process, the lab determined that the entire panel should not be analyzed when making a diagnosis, so the actual test differs slightly from the RainDance panel and only includes the exons plus 25 flanking nucleotides, she said.
The process becomes even trickier when trying to validate exome sequencing tests, since most exome capture kits do not cover the entire exome, said Chin. So, while Baylor's glycogen storage disorder panel can cover 100 percent of all targeted bases, an exome kit will generally miss between five to 10 percent of the target.
Labs that offer clinical exome tests based on exome-capture kits have to disclose which portions of the exome are not covered. If mutations are found in a low-coverage area, the lab will have to disclose that as well, Chin said.
And since there are currently no universal guidelines, "each lab will have its own policy for how it will disclose that," he said.
Validating cancer tests also poses unique challenges because samples will have varying tumor content and will come from various sources, including fresh frozen, fresh, and formalin-fixed, paraffin-embedded.
Shashikant Kulkarni, medical director of Genomics and Pathology Services at Washington University's School of Medicine, has said that clinical cancer tests have to produce the same results no matter what the cancer type (CSN 11/30/2011).
Validating Wash U's 28-gene cancer panel on the Illumina HiSeq took around one year, Kulkarni said.
A Brighter Future?
Chin said much of the validation steps that clinical labs currently have to take could be avoided if kit manufacturers took their assays through FDA 510 (k) clearance.
Currently, labs tend to launch multi-gene sequencing panels based on their areas of expertise, since so much of the development and validation work has to be done in-house, he said, noting that if labs had access to FDA-approved kits, they would not have to bear the burden of determining the specificity and sensitivity for every test.
FDA approval would "dilute the need to have experts in assay development be part of your team," Chin said.
Kits with FDA approval have been proven to work, he said, so labs can "just bring it in and teach people how to run the machine and how to interpret the data."
In the future, he said this is how kits will likely be developed, and said that there is "already a movement toward getting FDA approval," for next-gen sequencing tests. The fact that Illumina and Life Tech plan to seek FDA clearance for their benchtop sequencers this year is indicative of where the field will go, he added.
Have topics you'd like to see developed by Clinical Sequencing News? Contact the editor at mheger [at] genomeweb [.] com.