CHICAGO – Researchers at Stanford University and their colleagues have developed a novel computing system for diagnosing monogenic disorders that parses full-text scientific and medical articles rather than just abstracts to match genotypes and phenotypes to literature with more accuracy than previous methods.
AMELIE, which stands for Automatic Mendelian Literature Evaluation, combs through 29 million abstracts indexed by PubMed and further examines hundreds of thousands of complete journal articles to uncover information that relates phenotypes to known genetic variants. With natural-language processing (NLP) and machine learning, the system ranks the most likely variants in a patient's exome sequence for that patient's phenotype.
Even in "singleton" patients — those without available exome sequences of relatives — AMELIE put the causative gene at the very top of the list for 66 percent of 215 patients with an established diagnosis from the UK's Deciphering Developmental Disorders (DDD) project. Also, the system included the correct gene among the top 10 variants 90 percent of the time, according to a study that appeared in Science Translational Medicine last week.
The researchers were able to replicate their results in a retrospective study of 56 singleton cases from Stanford Children's Health and the Manton Center for Orphan Disease Research at Boston Children's Hospital. Here, AMELIE put the causative gene at the top of the list for 59 percent of the patients and in the top 10 for 89 percent of this test group.
AMELIE also far outperformed other technologies. For comparison, Exomiser, a matching system from the Wellcome Sanger Institute, had the causative gene in the top spot only 38 percent of the time, and Phen-Gen, another tool, got the top one right just 8 percent of the time, according to the paper.
"If using any of the other methods, the clinician would have to investigate between a median of 30 genes (when using Exomiser to rank patient candidate causative genes) and 108 genes per patient to arrive at the diagnosis in 90 percent of diagnosable cases," the authors wrote.
They also noted that if clinicians followed the rank order of candidate genes suggested by AMELIE, they would have to go through 735 gene-patient matches to find the causative gene for all 215 patients. However, if they went through the list of candidate genes in random order, they would need to evaluate 14,383 gene-patient matches to arrive at the causative gene for all patients.
AMELIE, they wrote, made getting to a diagnosis nearly 20 times faster than a random evaluation.
"We aimed to accelerate the diagnosis of patients with Mendelian diseases by using information from primary literature to construct gene rankings, thus allowing clinicians to discover the causative gene along with supporting literature in a minimum amount of time," the researchers wrote.
"Because AMELIE is an automatic curation approach requiring only an initial critical mass of human-curated data to train on, it is not constrained by the bottleneck of ongoing human curation," the paper said.
Corresponding author Gill Bejerano, an associate professor of developmental biology, computer science, pediatric medical genetics, and biomedical data science at Stanford, said that AMELIE not only saves scarce clinician hours but also helps physicians make more accurate decisions.
"The field that we're in is so hungry for an approach like this, because so much time is spent on literature evaluation," Bejerano said. By having a computer make the first pass, humans can devote their time to the 10 percent of cases that AMELIE cannot yet suggest a proper diagnosis for.
"The challenge is for the computer to handle the volume and to allow the human to be the seal of approval or a source of learning for the mistakes," Bejerano said. He added that AMELIE can also help with reanalysis of undiagnosed cases.
AMELIE, which has been online since 2017, builds a knowledgebase by applying NLP to millions of full-text articles. The architects trained the system on public resources including the Online Mendelian Inheritance in Man (OMIM), Human Gene Mutation Database (HGMD), and ClinVar.
A machine-learning engine then compares phenotypes and genotypes to the knowledgebase to produce a ranked list of candidate causative genes. Each result includes literature citations so humans can verify the recommendations.
Only a small portion of the 29 million articles that PubMed has indexed is relevant to diagnosis of monogenic diseases, so the researchers built a "classifier" that looks for potentially salient articles based on titles and abstracts. This filtered out about 98 percent of the total PubMed literature set.
AMELIE extracted several elements from full-text articles, including a median of three variants from each of the 123,073 papers it completely parsed, eventually building a knowledgebase of 872,080 gene-phenotype relationships involving 11,537 human genes. The system then filtered each patient variant list to identify "candidate causative variants" that are rare in those without a Mendelian disorder.
A metric dubbed the AMELIE classifier assigns a score of zero to 100 to "triples" of a set of phenotypes, a potential causative gene, and an article about the specific gene, with a higher score suggesting greater article relevance. From these scores, the technology constructs its list of probable causative genes.
Bejerano said that AMELIE advances science not only because it pulls facts from the full text of relevant PubMed-indexed articles, but also because it compares patient-specific phenotypes to information in the knowledgebase built from that NLP.
"We will pull up any patient that comes up, compare, analyze on the genomic side, and say, 'here are all the candidate genes in this patient'," Bejerano said. The system goes through close to 9,000 papers per patient. "Imagine if you have to have a human read through 9,000 papers for a patient," he said.
"The biggest innovation I think we showed there is that the two pieces can work in harmony. Our knowledgebase is not perfect, but when you couple it with real patients, the correct paper and the diagnosis come up to the very top," Bejerano added.
The study focused on pediatric Mendelian disorders, but Bejerano said that the method could apply to any monogenic disease. He has attempted to test AMELIE with adult patients, but a potential collaboration fell through for timing reasons. "There is no reason it shouldn't work as well with monogenic adult cases," he said. In addition, cancer is a "very relevant target" for AMELIE, he noted, without elaborating.
"You can really have the front line of diagnosis be done by machines today. I think that's what the paper unequivocally shows," Bejerano said. Automating diagnostic processes will grow in importance as the prevalence of sequencing increases, he added.
However, there are limitations because AMELIE relies on exome sequencing data rather than whole genomes, plus the technology is only as complete as current literature, which may not include a complete set of phenotypes.
AMELIE also is dependent upon the Human Phenotype Ontology (HPO) for training its AI. Although it picked up 80 percent of relevant full-text articles during the study, one-fifth of potential knowledge that does not include HPO terms to describe phenotypes may have fallen through the cracks, the authors acknowledged.
Peter Krawitz, director of the Institute for Genomic Statistics and Bioinformatics at the University of Bonn in Germany, said that he would consider using AMELIE in his own research, though he expressed some reservations.
"There clearly is a need for better tools in that area," he said. "The tools that are available are not perfect. Everyone knows that. Everyone understands that we need better support, but there are different approaches that research groups choose."
While the PEDIA researchers used some of the same comparison scores as AMELIE, including Phenomizer, the former is image-based, while AMELIE is completely text-based, so the two are somewhat complementary, Krawitz said.
Krawitz also serves as chief data science officer for FDNA, which sells facial recognition software called Face2Gene that combines facial traits with genomic and more traditional phenotypic data to assist in the diagnosis of heritable diseases. The PEDIA study in part assessed this technology.
Krawitz said that the PEDIA team found it too difficult and somewhat extraneous to assess entire text documents when searching for HPO terms, but he applauded the AMELIE team for taking on that task.
He noted that medical geneticists like himself have tended to write longer clinical notes than physicians in other specialties because cases can be complex, particularly in gathering family histories, but they are starting to change. "It's so time-consuming and often we don't even know whether the other doctors appreciate this length, so [it has become] a tendency to shorten it," he said.
"It's not necessarily an advantage if you can work with continuous text, because the field might actually move to more bullet list-like reporting," Krawitz said. "We assume that we can skip that part. We would rather focus on HPO term lists."
Krawitz praised the AMELIE study as "good work" but did urge caution because it was based on retrospective and simulated data. "Maybe [we will learn more] relevant information two years from now when they are able to benchmark it in a clinical setting," he said.
Bejerano said that follow-up work is underway.
In Bonn, Krawitz and colleagues are in the process of writing a follow-up prospective study on PEDIA that covered patients across Germany.
In the meantime, the Stanford team is working on making AMELIE available to others. So far, AMELIE has not used Stanford's high-performance computing infrastructure, just a server that Bejerano's laboratory bought and has been supporting. The hardware is in the process of being upgraded and the website is being redesigned; a week ago, the AMELIE website carried a disclaimer that the server was being overloaded with VCF file uploads from outside users who may have been trying out the technology.
The project has been chiefly funded by Bejerano's lab, but he is looking for an outside source to make AMELIE more sustainable and scalable.
"We are now exactly at the point where we are going to put a new piece of hardware and actually even an improved interface to allow people to upload their VCFs and do more of the [analytics] work on the website itself," Bejerano said.
"The hope is that this would be embraced in a way that [allows us to be] appropriately funded to facilitate all the hardware that the community would need, but that would really be depending on the community and how much support they put behind this," he said.