NEW YORK (GenomeWeb) – After releasing white papers, holding several public workshops, and listening to stakeholder feedback, the US Food and Drug Administration has released two draft guidances outlining how it might review the analytical and clinical validity of next-generation sequencing tests.
The FDA has tried to keep this work separate from the ongoing controversy surrounding its proposal to regulate laboratory-developed tests (LDTs). However, lab industry professionals fundamentally opposed to FDA oversight of activities and processes they consider to be the practice of medicine and beyond the agency's statutory authority aren't interested in softening their stance, not even after FDA Commissioner Robert Califf explained last week that regulation of genomic tests, particularly NGS, was a critical part of advancing the President's Precision Medicine Initiative.
During a funding update on the PMI last week, the FDA released a draft guidance outlining how test developers could demonstrate the analytical validity of NGS tests for germline diseases, and another draft guidance on how they might establish the clinical validity of genetic variants gauged by such tests using public databases recognized by the agency (see related story, published tomorrow). The FDA has emphasized that the draft guidelines present a "conceptual framework," not requirements.
The recommendations represent "a voluntary approach that harnesses communal data and, as FDA envisions, will soon be based on the community's own standards," FDA spokesperson Lyndsay Meyer told GenomeWeb. "This is not FDA imposing rules. It is a collaborative approach that builds on the expertise and best practices of the community."
Test developers are free to demonstrate analytical and clinical validity "by other means," Meyer explained, adding, "We believe all test developers could benefit from the best practices contained in our guidance whether they come in to FDA or not."
The majority of NGS tests are LDTs, and until the FDA finalizes its draft guidance on LDT oversight, labs don't have to submit them for review. But ever since the agency released its risk-based, phased-in plan for regulating LDTs two years ago, lab industry players have hired lawyers, lobbied congress, and written alternative legislative proposals to try to stave off the FDA and keep regulation under the traditional CLIA pathway.
The American Clinical Laboratory Association, the most vocal opponent of FDA oversight of LDTs, said in a statement last week that it "supports the spirit of the President's efforts to advance precision medicine," but is maintaining its stance that "LDTs are not medical devices, including those LDTs that utilize NGS technology." ACLA called on the Obama Administration to work with Congress and the lab industry to reform diagnostics regulations by updating CLIA.
Perhaps in an attempt to keep the LDT controversy from derailing its efforts to advance standards for NGS — a complex and powerful technology that is quickly dominating genetic testing — at public workshops to discuss regulatory approaches, the FDA requested participants not bring up the LDT issue. The agency even took to Twitter last week to distance the two draft NGS guidances from the LDT debate.
But this attempted separation seems contrived to lab professionals on the ground. "All of these next-gen tests are LDTs," Sherri Bale, managing director of GeneDx, told GenomeWeb. "You can't separate these. There's politics behind all this."
The draft guidances on NGS testing, she suspects, might be a diversion before the FDA finalizes the LDT guidance, which the agency has said it will do sometime this year. "This is out there to sort of side track us," Bale said. "If we accept next-gen sequencing regulation, we're accepting regulation of LDTs and then, why should we whine about anything else?"
Roger Klein, medical director of molecular pathology at Cleveland Clinic, also wasn't placated by FDA's assurances and pointed out that the draft guidances on NGS are directed at "stakeholders and FDA staff." The overly broad category of "stakeholders," in his view, does seem to incorporate LDTs "from a rhetorical standpoint." Even though the FDA insists the NGS guidelines are unrelated to the LDT guidance, Klein believes the agency has written the former "with an eye toward clinical labs."
Klein also chairs the professional relations committee of the Association for Molecular Pathology, a group that has lobbied against FDA regulation of LDTs. AMP is still looking over the NGS draft guidances, but the association's overarching position remains unchanged, he said, that FDA regulation of LDTs is not in the best interest of patients.
Despite having engaged with the agency through several public workshops on NGS regulation, from a practical standpoint lab directors cannot help but look at these guidances as a forewarning of the added responsibilities they will be subject to amid increasing regulation and tightening reimbursement.
Madhuri Hegde, executive director of Emory Genetics Lab, for example, noted that her lab already has to meet requirements to maintain CLIA certification, accreditation from the College of American Pathologists, and New York State Department of Health approval. Taking an NGS test through FDA review will require additional investments in personnel and resources that many labs don't have, which lab professionals fear will limit their ability to improve tests in step with rapidly evolving science.
"The external pressures to put all this together makes the price of the test go up, and we don't get reimbursed for that," Hegde told GenomeWeb. "Innovation is in all areas of genetic testing."
The FDA did get some credit for incorporating feedback from a wide variety of stakeholder groups into the NGS draft guidances. Daryl Pritchard, executive VP of science policy at the Personalized Medicine Coalition, lauded the FDA for working collaboratively and encouraged it to continue doing so.
"While others will certainly disagree, I sincerely think this represents a good faith effort by the FDA to be cautious, measured, and thoughtful, yet rigorous and comprehensive," said diagnostic regulatory and reimbursement strategy expert Girish Putcha. "If nothing else, if these documents are taken seriously and criticized constructively, not simply dismissed out of hand using the same old tired talking points, then I think and hope that they will help the field as a whole advance, and ultimately and most importantly, help patients."
What may get lost in all the usual consternation over FDA oversight of LDTs is that adherence to either of these guidances ... is strictly voluntary until and unless the FDA regulates LDTs.
A model approach
Perhaps because the FDA released the NGS draft guidelines while the fracas over LDT regulation is still ongoing, some industry observers opined that the agency decided to tackle the analytical validity of germline NGS tests first because it would be less controversial than taking on somatic tests. According to FDA's Meyer, however, the agency decided to use germline NGS tests "as a model to develop an overall approach," because the benefits and risks of such tests are better understood than other NGS diagnostics.
"In addition, we believe it would be more feasible to first evaluate the impact of the proposed standards-based approach for a subset of moderate risk NGS-based tests uses before expanding the approach to other intended uses," Meyer said.
In the analytical validity draft guidance, the agency noted that test developers may request Class II (moderate risk) designation for germline NGS tests using the de novo process. Moreover, the FDA may even exempt Class II germline NGS tests from 510(k) notification based on whether a test meets an FDA-recognized standard for analytical validity, such as the recommendations in the guidance, or other metrics advanced by the scientific community or a standards development body (SDO) recognized by the agency. For premarket notification exemption, the test developer would also have to provide assurance of the test's clinical validity and make test performance data publicly available.
However, the FDA also states in the draft document that it "is unaware of any existing, comprehensive standards" for analytical validation of NGS germline tests that could provide "reasonable assurance" of their safety and effectiveness. Lab industry professionals who have to abide by checklists from CAP and NYSDOH may not agree with this statement, but according to FDA's Meyer, NGS guidelines from those groups don't address product development.
Putcha interpreted the draft guidance language to suggest that these checklists technically may not qualify as "standards from an SDO," and that the FDA likely considers recommendations from these other groups as "insufficient." He believes there are legitimate reasons for why the agency might think so, mainly that checklists from CAP and NYSDOH focus on standards for lab processes, not test performance.
Although the FDA publicly releases a summary of the safety and effectiveness data on approved and cleared tests, these other entities provide "little to no transparency" about the labs and tests they evaluate, he reflected. Furthermore, "there is no requirement that a CLIA-certified lab be accredited by CAP, or that a lab that does not test NY patients get a NY permit, or its test approved by NYSDOH," said Putcha, who is director of laboratory science at Medicare contractor Palmetto GBA, although he was not speaking to GenomeWeb in that capacity.
Palmetto's MolDx program, which establishes criteria for the clinical utility of molecular tests, has issued analytical performance specs for detecting somatic variants using tissue DNA and for circulating tumor DNA tests. "To my knowledge, among the organizations cited, including CAP and NYSDOH, only the MolDx specifications articulate a similar, if less extensive, mix of process and performance standards," Putcha said.
He recalled that during the public workshops on NGS tests, professionals from academic and commercial labs indicated that a hybrid of general process and more specific performance standards were needed, and the FDA seems to have listened to that feedback.
"What may get lost in all the usual consternation over FDA oversight of LDTs is that adherence to either of these guidances for [an NGS tests that is] an LDT is strictly voluntary until and unless the FDA regulates LDTs," Putcha further reminded. "While others may have more Machiavellian conspiracy theories, seems to me that the FDA is trying to put forth its thoughts about analytical and clinical validity for such tests, and alternative regulatory paths, including exemption from premarket notification, that test developers can pursue."
The testing community would like to see a reference for that.
'In over their heads'
Although the FDA believes that the latest draft guidances are a byproduct of its collaborative engagement with stakeholders, lab professionals GenomeWeb spoke to found much to criticize in the document on analytical validity, particularly when it came to the minimum analytical validity performance thresholds set forth by the agency. For example, the FDA stated that the positive percent agreement, negative percent agreement, and technical positive predictive value should be no less than 99.9 percent with a lower bound of 95 percent confidence interval of 99 percent for all variant types the test reports.
Without being privy to the data on which FDA based these specifications, Putcha is not sure that this level of analytical performance is realistic for all variant types, since "the performance on larger indels and multi-exon deletions and duplications can be less robust than on SNVs and small indels."
Lab professionals had the most difficulty deciphering the following FDA recommendation: "For detecting germline heterozygous variants using a targeted panel, set a threshold of 20X or greater for minimum coverage depth and 300X for average coverage depth at 100 percent of the bases for targeted panels and at least 97 percent of the bases for [whole-exome sequencing]."
During a stakeholder call with the FDA to discuss the draft guidances, Bale asked the agency if the "300X average coverage" wording is a typo. Emory's Hegde and Heidi Rehm, director of the Laboratory for Molecular Medicine at Partners Healthcare Personalized Medicine, agreed that the coverage criteria articulated by the agency needs clarification.
"The 300x average coverage is worrying. Where did that come from?" Hegde wondered, pointing out that none of resources noted by the agency at the end of the draft guidance include this threshold. "The testing community would like to see a reference for that."
"I do not believe that it is necessary to achieve an average cover of 300X for a routine germline sequencing test," said Rehm, who added that the agency has told her in discussions that the statement was an error. She explained that although many labs run their targeted panels at coverage levels this high, it is not necessary unless there is quite uneven coverage in certain areas and the lab is raising average coverage to rescue problematic areas with much lower coverage, or if the lab is using NGS data to call a number of variants that can benefit from deeper coverage.
Typo or not, Bale interpreted most of the minimum performance thresholds proposed by the agency as "unnecessarily high" and as evidence of its lack of NGS knowhow. "They don't know what they're saying," she said, adding that test developers already have good guidelines from CAP and NYSDOH. "What does the FDA think it's going to add to this?" posited Bale, who is a founding member of the American College of Medical Genetics and Genomics, and an author on NGS standards issued by the group in 2013. "From a technical aspect, they are way in over their heads."
The lab community has 90 days to comment on the guidance. During that time, industry insiders GenomeWeb spoke to said they will communicate the problematic technical specifications in the draft document. But many also don't plan to back down from their position that the FDA regulates devices, which LDTs, including NGS tests, are not.
In advancing analytical standards for the design, development, and validation of NGS tests, the FDA is extending beyond its statutory authority into what pathologists and lab professionals consider the practice of medicine, Klein said. But he feels the agency especially over-reached in the draft guidance by issuing recommendations for how test developers should report results. "The FDA feels that even professional interpretation and communication are within its purview," he said. "Even for those that favor FDA regulation, this extends beyond what is considered a device."
This is the first of a two-part story on FDA's draft guidances on next-generation sequencing tests. The second part on draft guidelines for demonstrating the clinical validity of NGS tests using public variant databases is published here.