BETHESDA, Md. — Instead of working to remove regulatory hurdles to innovation, diagnostic developers should focus on meeting regulatory organizations’ evidence thresholds to ensure their tests are safe, effective, and clinically useful, the head of the Evaluation of Genomic Applications in Practice and Prevention working group suggested to participants at a meeting here this week.
“We have a national attitude that more is always better, that technology is always good. The optimism in this room is quite breathtaking in this regard,” Alfred Berg, chair of the EGAPP working group, said at a conference sponsored by the National Institute of Diabetes and Digestive and Kidney Diseases.
“I'd have to say that much of the language in the audience about removing barriers strikes me as odd. Some barriers are there for a reason. Evidence might be one of them,” he added, offering a glimpse into the mindset of a reviewer charged with the task of evaluating whether genetic techonologies currently on the market are appropriate for public use. “We have an environment that is hostile toward regulation. These tests have potential for benefits and harms, and unfortunately we have limited evidence.”
His comments come at a time when industry groups are criticizing the US Food and Drug Administration’s efforts to regulate a more complex subset of laboratory-developed tests, called in vitro diagnostic multivariate index assays, as a hindrance to innovation [see PGx Reporter 02-14-2007, 02-06-2008].
EGAPP’s recommendations are part of a pilot project developed in 2004 by the National Office of Public Health Genomics at the Centers for Disease Control and Prevention. The project aims to evaluate genetic tests and other genomic applications currently in transition from research to clinical use.
According to Berg, most of the genomic technologies that EGAPP has evaluated lack evidence to support their clinical utility and validity.
So far, the group has issued negative assessments of ovarian cancer detection and testing and CYP450 testing for SSRI-treated patients. The working group is planning to release three new sets of recommendations later this year about gene-expression profiling in breast cancer, genetic testing for lynch syndrome in colorectal cancer patients, and testing for UGT1A1 in colorectal cancer patients treated with irinotecan.
Initial assessments of gene-expression profiling in breast cancer show that while the technology suffers from a similar lack of clinical evidence, such tests’ potential benefits may outweigh potential risks to public health.
“We know very little about gene prevalence in general populations. We don't know that much about penetrance. We have very few clinical trials that compare testing and enrichment strategies,” Berg said.
“Many of the studies that we do have don't assess all relevant outcomes, many don't pay equal attention to benefits and harms, and virtually no one is talking about cost and feasibility,” he added.
The lack of evidence assuring the safety and utility of genetic tests is particularly problematic for primary-care physicians, since diagnostic companies are increasingly marketing directly to this group of doctors, suggested Berg, who also chairs the University of Washington’s Department of Family Medicine.
For instance, when Myriad Genetics last September launched a nationwide direct-to-consumer television ad campaign marketing its BRCA 1 and BRCA 2 breast cancer test, doctors worried that their offices did not have sufficient expertise or staff to handle the potential influx of request for BRCA testing [see PGx Reporter 09-12-2007, 10-10-2007].
After the BRACAnalysis Test ads began appearing, health officials also feared they may encourage women at low risk for developing breast and ovarian cancer to take unnecessary medical measures. Late last year, the office of Connecticut Attorney General Richard Blumenthal is reported to have received reports from healthcare professionals that BRACAnalysis has a potential for “misinterpretation and overreaction.” Blumenthal’s office has said it is investigating the accuracy of Myriad’s claims in its ads.
The three evaluations EGAPP is working on — about gene-expression profiling in breast cancer, genetic testing for lynch syndrome in colorectal cancer patients, and testing for UGT1A1 in colorectal cancer patients treated with irinotecan — are currently in draft form.
Of the three forthcoming recommendations, the one investigating gene-expression profiling in breast cancer is the furthest along. "We found limited evidence of analytic validity, limited evidence of clinical validity. Of course no direct evidence, meaning controlled trials, [on] clinical outcomes or clinical utility. There are mixed estimates of cost-effectiveness," said Berg, who also chairs University of Washington’s Department of Family Medicine.
“I'd have to say that much of the language in the audience about removing barriers strikes me as odd. Some barriers are there for a reason. Evidence might be one of them.”
A study published in the March issue of the American Journal of Human Genetics came to similar conclusions. The researchers reviewed the services of seven companies offering predictive genomic profiling by testing at least 69 polymorphisms in 56 genes.
“There is insufficient scientific evidence to conclude that genomic profiles are useful in measuring genetic risk for common disease or in developing personalized diet and lifestyle recommendations for disease prevention,” the authors concluded.
Unlike EGAPP’s evaluation, however, the AJHG study was not specific to breast cancer. And Berg hinted that despite the limited evidence supporting genetic profiling for this indication, “there is a very plausible positive balance with potential benefits versus potential harms.”
The EGAPP recommendations for gene-expression profiling in breast cancer will be out for peer review soon. The reports about genetic testing for lynch syndrome in colorectal cancer patients and testing for UGT1A1 in colorectal cancer patients treated with irinotecan are in earlier draft stages.
Earlier this year, EGAPP released its evaluation of using cytochrome P450 testing with Roche’s AmpliChip or other platforms to help dose depressed patients treated with selective serotonin reuptake inhibitors.
“In the absence of data that testing influences treatment or outcomes, there is a risk that the CYP450 test could increase costs without helping patients,” the group found.
Finding insufficient evidence to support routine CYP450 testing for this population, EGAPP discouraged doctors from conducting CYP450 testing on patients beginning SSRI treatment until the completion of further studies to confirm the clinical utility and validity of such tests.
Prior to its assessment of SSRI genetic testing, AHRQ released its report on ovarian cancer detection and management. For this report, EGAPP evaluated tests for single gene products, genetic variations affecting risk of ovarian cancer, gene expression, and proteomics for CA-125 and BRCA1/2.
Although AHRQ found no evidence suggesting genomic tests for ovarian cancer have adverse effects beyond those common to other ovarian cancer tests – which primarily include the risks of diagnosis for false-positive results and the risks of delayed or inappropriate treatment of false-negative results – “model simulations suggest that annual screening, even with a highly sensitive test, will not reduce ovarian cancer mortality by more than 50 percent.”
Berg noted that EGAPP’s challenge is that the group has to promulgate an entirely new framework for analysis when reviewing genetic tests, since the vocabulary, evidence requirements, and ethical issues for such technologies are markedly different from other forms of evidence-based medicine.
“We spent a great deal of time developing analytic frameworks, how we derive key questions, the search strategies that we use, the kinds of literature we look at,” Berg said, adding that the group has particularly struggled with one type of data, called “gray literature.”
This kind of data “is quite problematic – what you do with stuff that you know is there but hasn’t been published yet?” Berg posited.
Berg reminded conference participants that the terms “analytic validity,” “clinical validity,” and “clinical utility” are all unique to the genetics community. “The rest of evidence-based medicine doesn’t know what ‘clinical utility’ is. They really just talk about outcomes,” Berg noted. “So, we’ve been in this interesting translation project, trying to adapt and translate methods from evidence-based medicine more generally into this genetic domain.”
Following EGAPP’s assessment of CYP450 tests for SSRI-treated patients, AmpliChip marketer Roche criticized the working group’s methodologies. “We are disappointed that the study didn’t consider how CYP450 testing is currently being used by physicians actively treating patients, nor which selective serotonin reuptake inhibitors have a narrow therapeutic window, making them more appropriate for CYP450 testing,” a Roche spokesperson said [see PGx Reporter 01-09-2008].
Berg noted that EGAPP plans to outline its review methodology, particularly for evaluating clinical outcomes, in more detail in the three recommendations due later this year.