NEW YORK (GenomeWeb) – As liquid biopsy genetic tests have rocketed into clinical practice over the last handful of years, clinical researchers, test developers, and others have begun to pay increasing attention to a relative lack of tools to ensure assays are validated and performed according to common, or at least equivalent standards.
Last fall, for example, saw the launch of a consortium called the Blood Profiling Atlas in Cancer (BloodPAC) under the White House's Cancer Moonshot Initiative, aimed at aggregating and harmonizing data on current liquid biopsy tests, and on standardizing protocols for sample collection, preparation, and analysis for future test development.
"While many blood proﬁling platforms exist for research use only (RUO), questions remain regarding the performance characteristics and clinical validation of these platforms, and standard protocols for sample collection, processing, and analysis remain to be established. This information is a prerequisite to the design of clinical studies to demonstrate clinical utility … assay reproducibility, and provide support for [US Food and Drug Administration] approval and payer reimbursement," authors wrote in a paper describing BloodPAC this spring.
In parallel to BloodPAC (and to other efforts by academic groups), companies have also begun to highlight new products this year, recognizing growing attention and appetite for these tools.
Test validation (and even more so post-launch quality control and proficiency monitoring) may not be the most exciting aspects of diagnostic development and clinical genomics, but they are nonetheless a crucial part of both proving analytical accuracy, and eventually building a case for clinical utility.
Reference standards, or materials, serve as truth sets that can be used to demonstrate the performance of an assay, test its limits, check to make sure it is being performed correctly, or to precisely compare different tests to one another.
With a much longer history, cancer tissue tests also enjoy a wider variety of basic tools for this validation and monitoring work. In liquid biopsy though, the production of contrived materials that hew close enough to what actual circulating tumor DNA looks like has been a challenge.
Actual clinical samples, while ideal for proving the clinical validity of a test, aren't available in enough quantity, and don't offer the manipulability that makes contrived references so useful for testing different aspects of an assay's performance.
"Although plasma samples collected from cancer patients would be the ideal source of quality control materials in terms of biochemical properties, it is impractical to obtain a sufficient volume of plasma from one or a few cancer patients for quality assurance programs involving a large number of laboratories," Chinese University of Hong Kong researchers Jason Tsang and K.C. Allen Chan wrote in a discussion piece last week in Clinical Chemistry.
"Furthermore, the absolute or fractional concentrations of mutations in the plasma samples cannot be easily adjusted to test the performance of the assays at clinically critical levels, for example, at the limit of detection," the two added.
But creating synthetic references that closely mimic actual biological samples is really hard, researchers say, due to a number of factors including the fact that circulating tumor DNA is not fragmented randomly — something that can affect different test technologies differently.
"Locus-specific PCR assays would favor the amplification of longer DNA fragments over shorter ones … whereas adaptor ligation-based methods commonly used in next-generation sequencing platforms would have preference toward shorter fragments. Therefore, quality control materials with similar fragment size distributions as the circulating DNA … would be required to reflect the analytical characteristics of different detection approaches," Tsang and Chan argued in their article.
More broadly, "the technical challenges of replicating the clinical sample really reflect the field's challenges," explained Russell Garlick, chief science officer at diagnostics tools manufacturer SeraCare.
In other words, the fact that liquid biopsy testing is inherently difficult — due to the need to detect mutations that occur at a very low frequency compared to background DNA — means that creating materials to test the tests themselves is also a hurdle.
As discussion around goals for optimal reference standards continue, the last few months have seen a number of announcements by commercial firms of new products in this vein.
SeraCare itself launched a second-version ctDNA reference material product this month, which encompasses 40 somatic mutations in more than 20 genes, like BRAF V600E, KRAS G12D, and various EGFR alterations. The company's references combine a genomic DNA background with synthetic variant-containing DNA sequences at defined ratios — in this case across a range of allele frequencies down to 0.125 percent, the company said.
According to SeraCare, its references are produced using a unique technology that creates a DNA fragment size-distribution which more closely mimics native cfDNA than other shearing methods.
The company is hoping that it can develop references that are full commutable to clinical samples, but Garlick said it's not clear yet whether they've gotten there with this latest version. In an abstract shared at this year's American Association for Cancer Research annual meeting, the company reported that when it submitted its reference materials to several testing laboratories along with similar material that didn't use its special approach to shearing, "variant detection was superior in the novel reference material … using hybrid/capture-based assays."
This suggests "greatly improved commutability compared to existing materials composed only of sonicated DNA," authors from SeraCare wrote.
Earlier this year, Horizon Discovery also announced a new cell-line derived, mechanically sheared cfDNA reference standard set in synthetic plasma, which contains eight cancer-relevant mutations including BRAF, EGFR, KRAS, NRAS, and PIK3CA at 5 percent, 1 percent, and 0.1 percent allelic frequencies.
Horizon also offers the same panel of mutations on their own, as well as a reference sample set of 10 cancer-related structural variants.
According to the company the new synthetic plasma-spiked multiplex panel — like SeraCare's synthetic reference — allows users to evaluate across the full lab testing workflow, from DNA extraction or isolation through generating results.
Extraction, in particular, is increasingly being recognized as a source of significant variability from test to test or lab to lab, and thus a significant area of need for reference tools, Hannah Child, Horizon's Diagnostics Product Manager said this week.
Rather than using synthetic DNA, Horizon believes it can better mimic biological reality by using actual mutated cell lines, Child added, which is why the company's panel has been limited to eight genes so far.
"The alternative is to go more down a synthetic route and both have plusses and minuses, but especially in some bioinformatic pipelines, we have seen that the synthetic method doesn't perform as well," she added.
Academic groups also appear to be trying to tackle the challenge of creating reference materials ever closer to true clinical samples. For example, in their commentary last week in Clinical Chemistry, Tsang and Chan discussed another publication by a team of researchers in Beijing, who reported developing a set of synthetic cfDNA quality control materials that they argue even better emulate biological reality than do currently available products.
The team's method used site-directed mutagenesis and CRISPR/Cas9 to engineer DNA fragments carrying a variety of clinically important alterations, mixing them with micrococcal nuclease-digested HEK293 cell-line DNA to obtain samples with varying allelic fractions.
When the researchers compared next-gen sequencing results using these materials to those from actual patient-derived plasma samples, they achieved strong agreement, based on both the size profile of cfDNA fragments and on various quality control metrics of the resulting sequencing data, the authors wrote.
Overall, the field seems to be undecided still on what approach is optimal for creating a reference sample that is commutable to actual clinical specimens.
In discussions about liquid biopsy test validation in the wake of the FDA's first approval of a blood-based tumor test last year — Roche's PCR-based Cobas EGFR Mutation Test v2 — stakeholders have expressed different opinions.
Roche said, for example, that its discussions with the FDA led to a decision to use a sheared DNA approach, but others, like Guardant Health President and COO AmirAli Talasaz, have said that they believe sheared DNA and synthetic materials are both inferior to using cancer cell-line derived DNA with known mutation status spiked into cell-free DNA samples from young healthy donors.
"Roche paved the way for acceptance of contrived samples by the FDA, and [this approach] is also accepted by CAP/CLIA as adequate materials for validation," SeraCare's Garlick said this week. However, he added, Roche's validation work for its EGFR liquid biopsy test demonstrates the need for, and the value of commercial multiplex reference products.
At a recent forum hosted by the FDA and the American Association for Cancer Research, for example, Roche Molecular Diagnostics' Research Head Walter Koch said that in order to complete the studies that supported the FDA approval of its test, Roche spent "in excess of $2 million," both for mutant cell lines that the company could use to create contrived samples, as well as other normal and non-small cell lung cancer specimens.
"We had to buy some pretty large volumes, which is not easy to get from cancer patients. Where we did find those samples that had the mutations of interest, they were often at low prevalence and low concentrations, so we screened about a thousand samples to find 80 specimens… Most of those only had about 200 copies per mL or less. If you're using that do to spikes, you're not starting from a good point," Koch explained.
That's two million dollars for EGFR, Garlick said. "What about all the other variants? … Creating sample sets for all of them would be prohibitively expensive."
As a result, commercial multiplex tools like SeraCare's, Horizon's — and potentially others using techniques like those described in the Clinical Chemistry study this week — appear poised for increasing adoption as more labs seek to validate multiplex tests under CLIA, or decide to move such diagnostics through the FDA.
In ongoing stakeholder discussions, some have also called for the promotion of specific tools that can be used across labs to assess competing or similar tests, perhaps even as a requirement for coverage and reimbursement if not for regulatory approval.
Garlick stressed that reference materials are not only necessary for test development and validation, but also for ongoing quality control or proficiency testing.
Having a good reference allows you to do this," he said. "You can send out to all the labs claiming commercial launch, and [evaluate them side by side.]"
According to Garlick, at a recent meeting of the Foundation for Genetics Research attendees estimated that there are at least 24 labs now testing clinical liquid biopsy samples. "But there has been no [proficiency] testing yet," he said.