NEW YORK – Diagnostic firm Biodesix has added functionality to its Diagnostic Cortex platform to increase transparency and make the company's artificial intelligence (AI)-driven test results more easily explainable to physicians.
The company hopes to inspire greater physician confidence in AI diagnostic algorithms by allowing physicians to better see how an algorithm arrives at its results.
The Diagnostic Cortex is a deep learning platform that can be used with clinical data and any high dimensional -omics data. Biodesix currently uses the Diagnostic Cortex alongside several of its commercially available tests such as the proteomic VeriStrat, which assigns the likelihood of a good or poor prognosis to treatment outcomes among lung cancer patients based on an analysis of serum proteins with matrix-assisted laser desorption/ionization time-of-flight, or MALDI-ToF, mass spectrometry using genomic, transcriptomic, proteomic, and radiological data to determine prognosis and treatment of lung pathologies.
While machine learning methods have grown increasingly common in biological research, AI-powered diagnostic tests have been slower to enter the clinic. Biodesix attributes this at least in part to the "black box" dilemma inherent in many AI algorithms.
"The problem is that you put the data in the one side, and you have no idea how the black box, that is, how the AI algorithm is making a decision," Robert Georgantas, senior VP of research and translational medicine at Biodesix, said in an interview.
"You basically feed [an] algorithm an image, and [the algorithm] modifies the input into an abstract representation of the input data," explained Alexander Lachmann, assistant professor of pharmacological sciences at the Mount Sinai Center for Bioinformatics, who is unaffiliated with Biodesix. "That abstraction doesn't necessarily have any real world meaning."
Joe Chan, a physician-researcher with a background in computational biology at the Memorial Sloan Kettering Cancer Center, added that this sort of abstraction can conflict with a physician's need to be able to explain and defend treatment decisions.
"It's not really clear which features are being optimized [by AI] to make those predictions," Chan explained, "and which of those features are more or less important. You might be able to get away without explaining feature importance in applications like Instagram, where you're just applying filters to faces but in the space of clinical medicine, having a mechanistic understanding of feature interpretability is even more important when patients' lives are at stake."
Understanding the need for this transparency and how it might translate to a market opportunity, Biodesix is making an effort to shine light into this black box.
“I think it would be fair to say that greater transparency will help with adoption and uptake of our tests," Georgantas commented.
To that end, Biodesix recently added a tool called the Shapley Explainability Module to the Diagnostic Cortex. This module reveals the features, or molecules, being used throughout the platform's calculations and the contributions these features make to the phenotypes of interest such as aggressive lung cancers or treatment response, allowing physicians to backtrack through the AI's decision-making process.
The Diagnostic Cortex determines feature contributions through modified Shapley values.
Conceived of in the 1950s by Lloyd Shapley, these values come from a game theory technique used to determine how much individual players contributed to winning a match. In this way, one can assign MVP status or determine how to split winnings.
"We realized that that was really analogous to how our AI worked in looking at different proteins and the contributions they made to [an] outcome," Georgantas said.
One caveat in using Shapley values as originally determined is that the technique assumes that each variable acts independently, a situation rarely found in cellular biology.
"Knowing that biological features are commonly correlated, we’ve modified the original Shapley [method] to account for dependencies," explained Georgantas. "Just as the Diagnostic Cortex is an AI platform highly specialized to answer clinical diagnostic problems, the Biodesix version of Shapley values has been modified to specifically answer biologically based questions."
Incorporating modified Shapley values into the Diagnostic Cortex is the latest improvement to the platform, and the company is examining ways to incorporate it into both existing tests such as VeriStrat, as well as those making their way through the pipeline.
In September at the International Association for the Study of Lung Cancer’s 2021 World Conference on Lung Cancer, Biodesix presented data from a study involving 68 patients with advanced non-small cell lung carcinoma, who received immune checkpoint inhibitor treatment and VeriStrat testing. That study showed that VeriStrat effectively stratified patients by predicted survival outcomes.
Tests in the pipeline include the Primary Immune Response, or PIR, test, aimed at interpreting patient immune response to cancer, and the Risk of Recurrence, or RoR, test, designed to identify early-stage NSCLC patients at high risk of recurrence following curative resection. Both assays use the company’s proprietary deepMALDI mass spectrometry technology to measure proteomic signatures in serum. While PIR is a purely proteomic test, RoR is a multiomic test that also incorporates clinical parameters.
The company presented data at the AACR meeting in April, showing that the PIR test effectively targeted immunotherapy treatment strategies for NSCLC patients.
Biodesix has also begun using the Shapley Explainability Module with pharmaceutical partners and plans to present data from those collaborations at the Society for Immunotherapy of Cancer meeting in Washington, D.C.
Other near-term plans include the addition of a 52-gene NGS lung cancer test with a 72-hour turnaround time, and the expansion of partnerships and services related to COVID-19 testing.