By Turna Ray
A recent study challenging the accuracy of Genomic Health's reporting of breast cancer patients' HER2 status by an RT-PCR-based test has revived a longstanding debate about whether advanced diagnostic tools on the market today are truly an improvement over more established methods.
The study, published in the Journal of Clinical Oncology in October by researchers led by David Dabbs of the University of Pittsburgh Medical Center, tested the HER2 status of 843 patients across three labs by immunohistochemistry and fluorescence in situ hybridization and compared the results to HER2 status as assessed by Genomic Health's RT-PCR-based Oncotype DX. The study authors reported that of the 36 cases deemed HER2-positive by standard FDA-cleared IHC/FISH testing, Oncotype DX determined only 10 patient samples as positive, 12 as "equivocal," and 14 samples — or 39 percent of the patients who were previously thought to be HER2-positive by IHC/FISH — as HER2-negative.
Dabbs et al. concluded that HER2 reporting by Genomic Health's assay had overall agreement of 98 percent with IHC/FISH testing, but deemed this number "meaningless" due to the large number of HER2-negative patients (93 percent) and the small number of HER2-positive patients in the study cohort. The researchers further found that positive agreement between Genomic Health's test and IHC/FISH was less than 50 percent and concluded that Oncotype DX had an "unacceptable false-negative rate" when reporting HER2 status.
This is an assertion that Genomic Health contests. "We reviewed all the processes and the steps and the controls for these  cases," Steve Shak, Genomic Health's chief medical officer, told PGx Reporter. "We confirm that the HER2 cases in the report were reported accurately." In its own validation studies for its HER2 assay, Genomic Health has shown 95 percent overall concordance between its test and IHC and 97 percent concordance between its test and FISH.
This discrepancy illustrates the challenge facing medical professionals as more personalized medicine products come to market. Industry observers estimate that molecular and genetic testing comprises a $7 billion market growing at between15 percent and 20 percent per year. However, the faster the molecular diagnostics industry grows, introducing new tools that promise to reduce the uncertainty about patients' disease progression and drug response, the more the gap seems to widen between those advanced technologies and clinical practice.
In 1998, the US Food and Drug Administration approved Genentech's Herceptin for HER2-positive metastatic breast cancer alongside Dako's IHC-based HercepTest, making Herceptin the poster child for personalized medicine. Thirteen years later, however, many medical professionals still believe that when it comes to Herceptin there is no gold-standard diagnostic method for identifying best responders to the drug.
"It seems that controversy is ever present regarding the optimal approach to testing for HER2 status in breast cancer," John Bartlett and Jane Starczynski of the Ontario Institute of Cancer Research wrote in an editorial accompanying Dabbs' analysis in JCO. Clinicians and patients "need guidelines (provided by the American Society of Clinical Oncology-College of Molecular Pathology and other experts, for example) regarding the appropriate use of additional tests beyond conventional IHC and FISH."
Bartlett and Starczynski believe that there is currently insufficient evidence to support qRT-PCR as a substitute for HER2 testing by IHC or FISH. However, with further research, the authors opine that there may be value in developing novel molecular diagnostic methods based on qRT-PCR or other multiplex technologies to provide more definitive results to patients who receive an equivocal result by IHC and FISH.
ASCO/CAP guidelines recommend that healthcare providers determine the HER2 status for all invasive breast cancer patients either by IHC or by FISH testing. The guidelines further state that labs performing HER2 testing show at least 95 percent concordance with another validated test for positive and negative assay values.
Genomic Health's own studies show that HER2 reporting by Oncotype DX is robustly validated against IHC and FISH testing according to professional guidelines. Furthermore, the company maintains it has never marketed Oncotype DX's independent HER2 readout as a substitute for IHC and FISH testing to determine who should receive HER2-targeted drugs.
In a validation study published in JCO last year, researchers at Genomic Health, the University of California, San Francisco, and PhenoPath Laboratories found a high degree of concordance between FISH and Oncotype DX for gauging HER2 status, but said that the assay needs to be studied further before it can be used to make treatment decisions with Herceptin.
"These data show that RT-PCR is highly concordant with high-quality, central laboratory-performed FISH and may be of use to clinicians and pathologists who are uncertain about select HER2 assay results obtained by FISH and IHC," the authors wrote in the paper, but added that "assessment of HER2 status by IHC and/or FISH should continue as the approach for making [Herceptin] or [Tykerb] treatment decisions."
Oncotype DX is a 21-gene expression test that assesses the recurrence risk for women with HER2-negative, estrogen receptor-positive, early-stage breast cancer that hasn't spread to the lymph nodes. Studies have shown that post-menopausal women who have hormone receptor-positive breast cancer that has spread to the lymph nodes may also derive benefit from testing with Oncotype DX.
Estrogen receptor, progesterone receptor, and HER2 are part of the 21-gene RT-PCR panel that underlies Oncotype DX's recurrence assessment. In 2008, upon the request of its customers, the company began reporting ER, PR, and HER2 gene expression status separately with every Oncotype DX test performed.
A pathologist involved with the recent JCO study and an independent oncologist both told PGx Reporter that doctors don't normally send samples deemed HER2 positive by IHC or FISH for reanalysis by Oncotype DX. However, when patients receive discordant results by IHC and FISH and require a third test, or if oncologists are trying to figure out whether to treat patients with chemotherapy, they might send patients' samples to Genomic Health for analysis.
"In truth I do not know any oncologist who does not get HER2 and ER testing by standard means," ASCO president George Sledge told PGx Reporter. "To do so would be outside the standard of care, and … most hospital standard operating procedures require ER and HER2 testing on invasive cancers.
"I suspect that this implies that a few oncologists obtain [HER2 testing through Oncotype DX] as a tie-breaker when there is an indeterminate FISH result or [there is] heterogeneity in the HER2 testing, though this would just be speculation on my part," continued Sledge. "If this is the case, then the argument, such as it is, is over those cases that were tough calls to begin with, rather than the standard run-of-the-mill determinations."
Sledge said he holds no financial ties to Genomic Health. However, he was one of the authors of a 2008 study published in JCO comparing HER2 reporting by Genomic Health's Oncotype DX to IHC.
In Sledge's view, the discrepancy between Dabbs et al. and Genomic Health with regard to the accuracy of the RT-PCR-based test only highlights the difficulty of identifying a best-in-class HER2 test for healthcare providers.
"I don’t think there is a true 'gold standard' for these tests. Every study ever done shows divergent results and lack of concordance between expert pathologists performing HER2 and ER testing," Sledge reflected. "Pathologist A disagreeing with Lab B or vice versa doesn’t mean that A was right and B was wrong, merely that they disagree."
Over-reliance on Oncotype DX?
Researchers collected more than 800 patient samples analyzed by Dabbs et al. from patients at Magee-Women's Hospital, Cleveland Clinic, and Riverside Methodist Hospital in Columbus, Ohio. According to Rohit Bhargava, co-director of surgical pathology at Magee-Womens Hospital and a co-author of the JCO paper, only two out 19 labs invited to participate in the study did so, because the other centers did not have enough ER-positive, HER2-positive patient samples that they had submitted to Genomic Health for analysis.
Still, Dabbs and colleagues found that among the population who were HER2-positive by IHC/FISH but HER2-negative by Oncotype DX, three patients at Magee-Women's Hospital and two patients at Cleveland Clinic were not given Herceptin based on their Oncotype DX results. The fact that some patients were denied Herceptin based on Oncoytpe DX, even though the test is not indicated for such use, suggests that there may be some variability in the way oncologists are using the assay.
"If you speak with oncologists in the field, there is a sense in the community that there is an over-reliance on the Oncotype DX assay," Bhargava told PGx Reporter. "These are new tests that are [indicated] for something else. Initially Genomic Health wasn't reporting ER/PR/HER2. But when they started reporting these, then oncologists started getting confused about the information they had and the information [Genomic Health] reported."
In their study, Dabbs and colleagues raise concern that cases determined to be HER2-positive by FDA-approved IHC and FISH tests were found to be equivocal or negative by Oncotype DX, which has not been FDA cleared but is analyzed in a CLIA-certified lab. "That was the highlight of our paper: Not to over-rely on just one test, especially where the HER2 component is not reliably assigning the patients who are clearly HER2-positive by the currently used assays that have been shown to predict response with Herceptin-based chemotherapy," Bhargava said.
Bhargava believes that Genomic Health's marketing of its separate ER/PR/HER2 reporting may have had a hand in prodding some oncologists to use the assay to perform diagnoses for which it is not validated. Although Genomic Health does not claim that Oncotype DX should be used to make treatment decisions with HER2-targeted drugs, the company does state in its marketing materials that its test is highly concordant with IHC and FISH-based tests in gauging patients' HER2 status, which, according to Bhargava, may be enough to drive use of the test in this setting.
Bartlett and Starczynski in their editorial concede that some oncologists may be using Oncotype DX in settings not supported by treatment guidelines. "Dabbs et al. express concern that clinicians may be using the quantitative HER2 score supplied by Genomic Health to substitute for in situ analyses and indeed suggest that this may have occurred for at least one patient during their study," they wrote. "A careful review of the data provided in several independent reports (including that cited on the Genomic Health Web site) shows that this concern may be valid."
[ pagebreak ]
In the opinion of Dabbs et al., the best way to establish the HER2 status of an invasive breast cancer patient is to combine both IHC and FISH. Bhargava added that labs should put in place quality assurance measures to ensure that unusual test results are further investigated.
"We have to question the results if they are not making sense. If it's a moderate-risk kind of tumor and the result is intermediate or [at the] low end of [the] high score, then it's making sense," he said. "But when the results are at the other end of the extreme we have to consider what went wrong."
At Magee-Women's Hospital, all patient samples sent for testing by Oncotype DX are reviewed by the lab director. For the JCO study, those patients who were found to have discordant results by IHC/FISH and Oncotype DX had their samples retested by IHC and FISH again to ensure that tumor heterogeneity wasn't confounding test results.
"Any time we send tissue out, we are always checking for quality assurance purposes that what we have said makes sense or not. So, all lab directors have an obligation to keep a quality check on whatever specimens they send out. We review slides from outside institutions and we send our slides to outside institutions, so whenever the report comes back we have some kind of correlation statement whether the outside institution agreed with us or not," Bhargava said. "This is an ongoing quality assurance process that all lab directors should have in place."
Dabbs and colleagues suggest in their paper that a false-negative result by RT-PCR assays can occur when there are low amounts of invasive tumor, or when the sample is diluted with other types of tissue, such as normal breast tissue. The study authors conducted a histological review of the discordant samples and found that most did not contain more than 50 percent of invasive carcinoma in the tissue block sent for testing by Oncotype DX.
However, Genomic Health maintains that the company performed the necessary manual microdissection prior to RT-PCR to ensure that non-tumor tissue did not reduce the HER2 measurement.
For his part, Shak posited that there may be a small number of cases where DNA amplification, RNA levels, and protein expression are truly discordant. "Many people in the field have commented that the group of patients that Dabbs has looked at may be enriched with cases in which HER2 expression [is] near the cut-off or with a weaker signal where the discordance rate might be a little bit higher," Shak said.
ASCO and CAP indicate in their guidelines on HER2 testing that approximately 20 percent of current HER2 testing with IHC and FISH may be inaccurate. "When carefully validated testing is performed, available data do not clearly demonstrate the superiority of either IHC or in situ hybridization as a predictor of benefit from anti-HER2 therapy," the professional guidelines state.
Last year, at the San Antonio Breast Cancer Symposium, researchers from the Mayo Clinic, Genentech, the National Cancer Institute, and elsewhere compared HER2 results from three laboratories and found that the results by IHC and FISH were discordant 8 percent of the time, though these discrepancies could be resolved via face-to-face adjudication.
Given that testing by IHC and FISH can also lead to contradictory HER2 results, Genomic Health's Shak took particular issue with the charge that Oncotype DX's false-negative rate is a cause for concern. He suggested that Dabbs and colleagues conduct another independent analysis to figure out which results are actually correct and what might have caused the discordance.
"The issue of discordance with HER2 testing between labs is well known from the very beginning," said Shak, who was one of the lead investigators in the pivotal trials for Herceptin. Dabbs et al. "use the term false-negative … The use of the term false-negative implies that one laboratory can call itself the HER2 gold standard and 100 percent accurate.
"I don't think [Dabbs] is the only pathologist that sometimes believes that, but most would not agree that one's work should be thought of that way, that it should not be questioned," Shak said.
Bhargava didn't appear too willing to accommodate Shak's request that he and his colleagues conduct yet another study to get further insight on the 14 cases his team deemed false-negatives. "The title of our study is an independent study. And that's what it means," he said. "I don't know why [Genomic Health has] such a hard time understanding that. This is not just our lab, there are two other labs" that got the same results.
This disagreement draws a sharp line between oncologists and pathologists in the era of molecularly guided medicine. Most of the authors of the latest JCO paper, including Dabbs and Bhargava, are pathologists, but Genomic Health markets Oncotype DX mainly to oncologists. According to the company, Oncotype DX is performed by physicians in 60 countries who have ordered nearly 250,000 Oncotype DX tests to date.
"To my knowledge Dabbs did not have any clinicians join him in his publication," Shak noted. "When I talked to the medical oncologists at his institution … the issue raised by the authors was not felt to be relevant to the way the test is actually being used to guide chemotherapy use."
If oncologists decide to test patients with Oncotype DX, the sample is shipped to Genomic Health's lab in California for analysis. Pathologists used to analyzing breast tissue on a morphological and molecular level have expressed discomfort with the "black box" nature of the Oncotype DX test, since the multi-gene expression algorithm underlying the test is proprietary to the company.
In Bhargava's view, looking at a single-gene product, such as HER2 gene expression reported by Oncotype DX, is a step backward from how pathologists currently determine the HER2 status of a sample. As a pathologist, "you are looking at so many different components within the breast tissue – the normal stroma, the normal ducts, lymphocytes, macrophages. Using morphology-based assays, I'm down to the level of a single tumor cell. I can tell you what the tumor cell is and what is the lymphocyte, and I can check the expression right there."
Although Bhargava sees utility in using a gene-expression test in a multiplex format, he cautioned that healthcare providers should not rely solely on the results of non-morphological tests such as RT-PCR. With such tests inaccurate results can sometimes occur if the patient sample has a mix of invasive and other type of tissues.
"When you're looking at a breast section from a pathologist's perspective, you're looking at so much, not just the invasive cancer, [but] you have to report what's in the invasive cancer … not the other stuff," he explained. "You're diluting that tumor with other stuff if you're using non-morphology based assays. That's my concern."
Dabb et al. ultimately suggest that Oncotype DX's false-negative rate for HER2 results raises questions about the predictive and prognostic capabilities of the overall recurrence score. Much of this concern is perhaps due to the "black box" nature of Oncotype DX. Pathologists may be able to compare IHC/FISH tests results in their labs with Genomic Health's HER2 reports but they cannot similarly investigate all the markers on the gene-expression panel to double check the recurrence risk provided by the test against their own measures.
Noting that the Oncotype DX recurrence score has been shown in the literature to be "heavily influenced by tumor hormone receptor content, HER2 status, tumor proliferation index, and/or grade," Bhargava expressed concern that an inaccurate HER2 result could impact the overall recurrence score. He added that Oncotype DX was truly a black-box test until Genomic Health began separately reporting ER, PR, and HER2, at which point pathologists had something against which to compare their own lab results.
"Laboratory professionals would certainly like more transparency," Bhargava said. "Another component that we as pathologist are interested in is the proliferation gene score, but currently that is not available."
Shak dismissed as baseless Dabbs and colleagues' charge that the Oncotype DX recurrence score is anything less than robust. "There have been multiple studies that we did to validate the assay, with outcome data from six large, well-designed, rigorous clinical trials," he said. "When you put it in that context, the speculation by the authors regarding the clinical validity of the recurrence score is unwarranted."
Meanwhile, as molecularly guided personalized medicine continues to advance, it is a good bet that there will be many more tests that use proprietary, multimarker algorithms to make prognostic and predictive claims.
"If the pathologists are concerned about the 'black box' aspect of this, I would be concerned, because there are many tests performed elsewhere that they have no real control over or way of counter-testing," ASCO's Sledge noted. "That’s just the nature of modern medical care."
Have topics you'd like to see covered in Pharmacogenomics Reporter? Contact the editor at tray [at] genomeweb [.] com.