NEW YORK – A study of health record data for more than 450,000 opioid-exposed veterans determined that the algorithm underlying a US Food and Drug Administration-approved assay fails to meet "reasonable standards" for efficacy in predicting opioid use disorder (OUD) risk and that its clinical use would result in high rates of both false positives and false negatives.
The case-control study, led by researchers at the University of Pennsylvania Perelman School of Medicine and published today in JAMA Network Open, calculated that the 15 candidate single-nucleotide polymorphisms (SNPs) contained in Solvd Health/AutoGenomics' AvertD assay correctly classified approximately 52 percent of individuals diagnosed with the disorder, which the study authors described as "slightly better than random guessing."
The study involved health record data from over 450,000 participants with opioid exposure in the Million Veteran Program (MVP), including more than 33,000 individuals with OUD.
Using logistic regressions, the researchers determined that the assay's 15 SNPs accounted for approximately 0.4 percent of variation in opioid use disorder risk, while age and sex alone accounted for slightly over 3 percent.
Compared to the 52 percent accuracy of the AvertD algorithm, estimating OUD risk by age and sex alone correctly identified close to 60 percent of cases.
The AvertD model's predictive power declined when applied to separate genetically inferred ancestry groups. Seven SNPs were found to be significant in the European-descent group, for example, while two were deemed significant among individuals of African descent, and only one in the admixed American descent group. Among individuals of African and of admixed American descent, the algorithm's SNPs accounted for only 0.04 percent and 0.16 percent of variance, respectively.
To test how an extreme case of model-embedded bias by population stratification could make the model falsely appear to produce meaningful risk prediction, the researchers selected a random sample of cases from one ancestry group and controls from another, artificially creating groups that reflected ancestry independent of genetic risk for OUD. This showed a clear separation of ancestry groups, whereas the standard model failed to discriminate between cases and controls.
"Thus, in a group of different ancestries, using the SNPs would differentiate ancestry groups and would be misinterpreted as differentiating OUD risk groups," Henry Kranzler, professor of psychiatry at the UPenn Perelman School of Medicine and senior author of the study, said via email.
The US Food and Drug Administration granted AutoGenomics premarket approval for AvertD late in 2023. This approval was followed a few months later, however, with a call from experts for the FDA to reverse its decision and for the US Centers for Medicare and Medicaid Services (CMS) to deny it coverage.
AutoGenomics was acquired in 2019 by Solvd Health, which was itself previously called Prescient Medicine.
"Solvd Health is confident in the clinical validity and rigor of AvertD, the first FDA-authorized genetic risk test for opioid use disorder, developed to empower informed prescribing decisions for acute pain," Ron McCullough, senior VP of clinical operations at Solvd Health, said in an emailed statement.
"The researchers in the recent JAMA publication did not have access to our technology (AvertD); therefore, any comparisons or conclusions in the article to AvertD are invalid," McCullough added. "The research relied on non-validated data, biased study populations, and methods inconsistent with established research and clinical standards, limiting its applicability. These limitations undermine their study's conclusions, which contrast sharply with the robust validation and regulatory review behind AvertD. Additionally, the researchers acknowledge numerous conflicts of interest, calling into question their motivations to publish this study. We encourage independent experts to evaluate these discrepancies and remain committed to advancing proactive healthcare solutions addressing the opioid crisis."
The study notes that Kranzler is a member of advisory boards for Dicerna Pharmaceuticals, Sophrosyne Pharmaceuticals, Entheon Biomedical, and Clearmind Medicine; and a consultant to Sobrera Pharmaceuticals. In addition, Kranzler and coauthor Joel Gelernter of the Yale University School of Medicine hold US Patent No. 10,900,082 titled "Genotype-guided dosing of opioid agonists."
The study's findings "underscore the need for more robust and complete data, particularly given the complex nature of psychiatric conditions, including OUD," Kanzler said. "The potential harms deriving from a faulty genetic test for OUD include both false negatives and false positives."
For example, Kranzler and his colleagues wrote that false positives can contribute to stigma surrounding addiction, causing patients undue concern, and potentially biasing other healthcare decisions. False negatives, meanwhile, might give patients and prescribers a false sense of security regarding opioid use and lead to inadequate treatment plans, particularly with respect to pain management.
The authors wrote that while AvertD is meant to complement standard clinical assessments, its use appears unlikely to confer additional benefits and may instead give providers and patients false and potentially harmful information.
The UPenn scientists further wrote that despite limitations such as reliance on electronic healthcare records that may be susceptible to bias and genetically inferred ancestry descriptors not always aligning with current guidance, the issues identified in their study "suggest that the manufacturer has a fundamental misunderstanding of genetic principles, particularly the impact of differences in population structure and allele frequency."
"Because genetic risk models in psychiatry will continue to emerge and could prove clinically useful," the authors wrote, "it is crucial that researchers and regulatory agencies adopt rigorous standards for developing and evaluating them prior to their application in clinical settings."