NEW YORK (GenomeWeb) – Next-generation sequencing-based cancer assays are increasingly used to look for genetic alterations in the tumors of late-stage cancer patients that may suggest additional treatments, but their widespread use in precision medicine will require them to be robust and reproducible across laboratories.
Providing an example of how such assays can be established, researchers involved in the National Cancer Institute's Molecular Analysis for Therapy Choice (NCI-MATCH) trial recently published the validation of their NGS assay in the Journal of Molecular Diagnostics, a process that included review by the US Food and Drug Administration. In particular, the developers had to make sure the assay results were reproducible across four different sequencing laboratories and they could analyze tissue samples collected at hundreds of sites.
"This is a well-designed study and it is valuable as it provides a template for the development and validation of NGS-based clinical assays," said Michael Berger, associate director of the Marie-Joseé and Henry R. Kravis Center for Molecular Oncology at Memorial Sloan Kettering Cancer Center, who was not involved in the validation study. He and his team developed a similar assay, MSK-IMPACT, that is used internally at MSKCC and has been cleared by the New York State Department of Health, based on a validation study published in 2015.
The NCI-MATCH study "reaffirmed that when standard laboratory protocols and analytical pipelines are utilized, NGS-based clinical assessment is highly robust and reproducible," Berger said. "It also demonstrated the importance of collaboration among clinical laboratories, particularly in the exchange of shared reference samples for benchmarking assay performance metrics."
The NCI-MATCH trial, which opened for enrollment in August 2015, aims to determine whether treating patients based on molecular aberrations in their cancers is effective. The trial, which is led by the ECOG-ACRIN Cancer Research Group, an organization that designs and conducts biomarker-driven cancer research studies, is open to adults with advanced solid tumors, lymphomas, or myelomas who no longer respond to standard therapy. It is offered at clinical locations across the US that participate in the NCI's National Clinical Trials Network, so it could involve up to 2,400 sites.
As the first step of enrollment, the NCI-MATCH assay screens a patient's tumor biopsy for more than 4,000 genetic variants across 143 genes. The assay amplifies the target DNA using Thermo Fisher Scientific's Oncomine Cancer Panel and sequences it on the Ion PGM. If patients have an abnormality in one of a small number of genes that can be targeted by a drug, or a "match", they can consider enrolling for treatment with that drug.
The trial initially started with 10 treatment arms — meaning it was looking for 10 types of gene alterations linked to drugs, such as EGFR-activating mutations and the drug afatinib (Boehringer Ingelheim's Gilotrif) or MET amplification and the drug crizotinib (Pfizer's Xalkori) — but it expanded to 24 treatment arms last May, after enrollment was paused for about six months to allow for scientific review.
Treatment arms exclude patients with tumor types for which a specific gene alteration is already standard of care. For example, patients with EGFR-activating mutations cannot enroll in that treatment arm if they have small cell or non-small cell lung cancer or a history of interstitial lung disease.
One reason the number of treatment arms, as well as the total number of patients to be screened, was expanded after the review was that the percentage of matching patients was lower than expected. Instead of a third of patients, the organizers said last May that they now only expect 23 percent of patients to match one of the 24 treatment arms.
Following the timeout — which the study took after almost 800 patients had enrolled within the first three months of the trial, many more than expected — NCI-MATCH increased the overall number of patients to be screened from 3,000 to 5,000. Since then, the number has grown to 6,000 patients, who the researchers expect to finish screening by May of this year. To cope with this many samples, lab capacity was expanded as well, so more than 100 patient specimens per week can now be processed. As part of this, the sequencing labs switched from the Ion PGM to the Ion S5 platform.
Physicians were also allowed to submit archived biopsy samples instead of fresh samples if they were no older than six months and if the patient had not received a targeted treatment since.
As of late February, the trial had obtained biopsy samples from approximately 4,500 patients, and about 4,000 patients had received their genomic test results. Of those, around 700, or 18 percent, had been matched to one of the 24 treatment arms.
According to Barbara Conley, a medical oncologist and associate director of the Cancer Diagnosis Program in the Division of Cancer Treatment and Diagnosis at NCI, 72 percent of matched patients have enrolled for treatment, which is higher than expected – usually only about 50 to 60 percent of eligible patients, she said, get on a treatment in other collaborative group trials. "We're very happy about that," she said.
Still, several treatment arms are far from meeting their recruitment goal. According to data from the ECOG-ACRIN website, as of mid February, two treatment arms had not accrued a single patient, and six others had fewer than 10 patients, while only 11 arms had filled at least half the patient slots.
Assay validation
Prior to designing the NCI-MATCH assay, the team sought advice from other clinical laboratories working in somatic cancer testing, asking them about their approaches and the platforms they use, said Mickey Williams, scientific director of the Molecular Characterization and Clinical Assay Development Laboratory at Frederick National Laboratory for Cancer Research. Based on their feedback, and on the fact that they needed to analyze formalin-fixed fresh biopsies, which often yield little DNA, they selected their assay technology and sequencing platform.
They also realized that they would need several clinical laboratories to run the assay, and that they needed to demonstrate that these could obtain identical results from the same samples. Four clinical molecular diagnostics CLIA labs were chosen as sequencing laboratories — at Frederick National Laboratory for Cancer Research, Massachusetts General Hospital, the University of Texas MD Anderson Cancer Center, and Yale School of Medicine.
Because the NCI-MATCH assay is a requirement for enrolling patients into the trial and is used to assign them a treatment based on their biomarkers, the team had discussions with the FDA about its validation plans and how it intended to use the assay, in accordance with the agency's abbreviated investigational device exemption process. "They worked very well with us and gave us some very valuable input," Williams recalled.
While the assay is regarded as an investigational device for the study, "it was considered non-significant risk because the biopsies carry very little risk of severe complications and the treatment that they guide would be commensurate with the risk patient would have whether or not the test was used," Conley explained.
To make sure all four laboratories would conduct the assay in an identical manner, they all met at Frederick National Laboratory for a week-long training. The standard operating procedures were finalized at that time, based on input from the participants.
A critical point was the use of a common, locked data-analysis pipeline, Williams said, because the parameters for analyzing next-gen sequencing data can be set in many different ways. The goal was to make the assay as specific as possible in order to prevent assigning patients with false positive results to a treatment. "We knew we would take a bit of a hit with the sensitivity," Williams said.
Another critical component was to evaluate and standardize the sample collection process, given that biopsies could technically be sent in from 2,400 sites across the country, most of them community cancer centers.
All samples are sent to a single location — MD Anderson Cancer Center — where the nucleic acid is extracted and RNA is turned into cDNA. The material is then passed on to the four sequencing laboratories. "We did a study using surgical specimens taken from MD Anderson and sent them through the entire process, including even shipping the specimens overnight to ourselves, to test the entire pathway from the acquisition of the tissue through the analysis," including sequencing and immunohistochemistry, said Stan Hamilton, head of pathology and laboratory medicine at the University of Texas MD Anderson Cancer Center. "That identified some elements that we hadn't quite frankly thought about in advance and contributed to the success that we've had with the assay."
For example, it became quickly apparent that a common sample collection kit was required that had all necessary supplies inside and was temperature controlled, so that all samples would be treated the same, Hamilton said. In addition, the team developed a set of criteria for what types of lesions are best for a biopsy. Another lesson learned was that it was helpful to have a single contact point for all sites sending in specimens, he said, so everyone got information from the same source.
Even though the SOPs for the assay have been locked, it is still possible to make modifications. The original design was driven by the drugs that were available for treating patients at the time, Hamilton said, but the plan is to add a number of new treatment arms, and a new version of the assay will come online in a few weeks. "We want to change as little as possible, obviously," he said. "We've added some components to it to take into account the new arms — it's not a major change." Validation across the laboratories will be replicated, however, with the new assay version.
Going forward, it will be important to show not only that a particular cancer assay provides reproducible results across several laboratories but also that different platforms produce consistent results. "We are going to have to demonstrate somehow that all next-generation sequencing platforms can get similar results, and I think that's the next big challenge for us as a community in the laboratory space," Williams said.
A recent side-by-side comparison of two cancer assays, for example — from Foundation Medicine, which tests tissue samples, and Guardant Health, which assays circulating tumor DNA from blood — found that the two yielded discordant results, but that might have been in part due to differences in the samples.
The answer will probably not be to converge on a single platform for all clinical cancer assays, but to choose the right assay for a particular purpose.
"I don't think it's reasonable to force everybody to use a single platform," Conley said. "But we do have an interest in making sure that whatever platform is chosen is adequately validated for the purpose it will be used."
"Every one of these technologies, and sequencing platforms, has advantages and disadvantages to it, and there is no perfect platform," Hamilton said, and choices need to be made "on the basis of what you are trying to accomplish," given the types of samples available and the targets to be covered.