NEW YORK (GenomeWeb) – Clinical laboratories participating in a pilot project to test recent guidelines for classifying genetic variants initially agreed only 34 percent of the time, a number that improved to 71 percent after discussions by phone or email. In about 5 percent of cases, the remaining differences could affect patients' medical management.
The results of the pilot, published today in the American Journal of Human Genetics by members of the Clinical Sequencing Exploratory Research (CSER) consortium, highlight the challenges genetic testing laboratories face in determining whether variants identified by ever-growing numbers of genetic tests contribute to disease, and how data sharing and collaboration can help.
According to the authors, the findings show that "classifying sequence variants is similar to other fields in medicine in which practitioners can legitimately differ in their assessments of pathogenicity of a laboratory finding." But using formal criteria to analyze the evidence, they wrote, helps to understand differences in expert opinion and reduces errors and discrepancies.
For the study, nine laboratories participating in the National Institutes of Health-funded CSER consortium interpreted a total of 99 variants, either using their own in-house methods or guidelines published last year by the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. The guidelines, which are intended for high-penetrance variants associated with Mendelian disorders, define 28 criteria to weigh as evidence, such as population frequency, case-control data, functional data, computational predictions, allelic data, and segregation data.
The variants — each laboratory contributed 11 — were either single-nucleotide substitutions or small indels in genes tied to Mendelian diseases that had come up in sequencing projects of the nine labs. They included variants classified by the submitting lab as pathogenic, likely pathogenic, of unknown significance, likely benign, and benign, and could be either diagnostic results or incidental findings.
Nine of the variants were interpreted by all nine laboratories, while the other 90 were randomly distributed between the labs so at least two laboratories would classify them.
Internally, laboratories' interpretations were mostly consistent, whether they used their own method or the ACMG-AMP guidelines — their classifications matched for 79 percent of the variants. For 5 percent of variants, the two interpretations by the same lab differed in a manner that might affect medical management, meaning that one classified the variant as pathogenic or likely pathogenic, and the other as a variant of unknown significance, likely benign, or benign variant.
Interpretations differed significantly between laboratories, no matter which approach they used: labs only agreed for 34 percent of the variants. For 22 percent, the differences were so significant that they might lead to different medical management.
To try to reconcile those differences, the group decided to discuss the 66 variants where interpretations between labs differed, either in a conference call or via email. After all evidence was discussed, which sometimes included a laboratory's internal data, each lab was asked to provide a final classification for the variant in question.
Following the consensus discussions, agreement between labs increased from 34 percent to 71 percent of variants, and only 5 percent had differences large enough so they might lead to alternative medical management.
One reason the discussions increased consensus is that they revealed in some cases that labs had not used the rules laid out in the ACMG-AMP guidelines appropriately. Other times, the talks led to expert judgment overruling other, conflicting evidence, such as computational predictions.
Overall, the authors concluded that given the complexity of the data and uncertainty about the validity of some if it, "it is unsurprising that there would be variation among laboratories regarding these determinations."
And while the use of the ACMG-AMP guidelines did not per se increase agreement between labs, their use "enhances transparency and facilitates resolution of discrepancies in variant interpretation," they wrote.
Part of why labs differed in their interpretation even though they used the same guidelines was "the subjective process of deciding when certain criteria are met," the authors wrote, but the guidelines also "provided a valuable framework for subsequent discussions of evidence, often leading to resolution of differences in variant interpretation," which would have been difficult if each lab had relied on its own method.
Overall, the study "identified areas of confusion regarding the ACMG-AMP criteria," which will be useful for developing training materials. The Clinical Genome Resource (ClinGen) consortium is currently developing tools to help labs classify variants based on the ACMG-AMP guidelines, they added.