A new machine-learning model achieved higher diagnostic accuracy on Mendelian disorders than existing bioinformatic tools
1. AI-MARRVEL (AIM), a new artificial intelligence (AI) system, was trained using samples diagnosed by experts from the American Board of Medical Genetics and Genomics. The system was subsequently tested on three patient data sets.
2. Compared with four existing top-performing algorithms, AIM doubled the number of solved cases. AIM achieved a precision rate of 98% and identified 57% of diagnosable cases.
Evidence Rating Level: 2 (Good)
Study Rundown: Mendelian diseases are caused by one or a few variants in a single gene, but identifying these variants is time-consuming and requires a broad knowledge base. A cost-effective solution is bioinformatics gene analysis, but existing tools have limited accuracy and were developed using simulated data. Mao and colleagues developed a new AI system – AIM, using 3.5 million variant data points. It was engineered using genetic principles and the clinical expertise of experts. Then, AIM’s accuracy against four existing, top-performing bioinformatic tools was tested using data from three distinct patient groups. The study found that AIM outperformed all four comparators across all three datasets. As the volume of training samples increased, AIM’s accuracy increased from 54% to 66% after incorporating additional engineering features. However, like other tools, AIM performed worse for cases with a recessive inheritance pattern compared to those with a dominant pattern. AIM was then further modified to produce a model dedicated to diagnosing recessive cases, achieving an accuracy of 63.4%. Similarly, additional training and filters enhanced its performance in diagnosing heterozygous variants. Overall, this study demonstrated the potential of AIM in identifying genetic variants for diagnosing Mendelian disorders and its superiority over existing algorithms.
Click here to read the study in NEJM AI
Click to read an accompanying editorial in NEJM AI
Relevant Reading: Open-Source Artificial Intelligence System Supports Diagnosis of Mendelian Diseases in Acutely Ill Infants
In-Depth [cross-sectional study]: Mao and colleagues developed AIM using 3.5 million variant data points from samples diagnosed and selected by experts certified by the American Board of Medical Genetics and Genomics. AIM was engineered with relevant knowledge such as minor allele frequency, variant impact, inheritance pattern, phenotype matching, gene constraint, etc., as well as different aspects of genetic diagnosis techniques. The authors compared AIM’s performance to four top-performing benchmarking algorithms: Exomiser, LIRICAL, PhenIX, and Xrare across three independent data sets (1377 total patients). Throughout the study, AIM was modified to create additional models to assess changes in its performance in specific diagnostic contexts (e.g., diagnosing recessive disorders).
AIM achieved higher accuracy rates than all four comparators across all three data sets, having diagnosed 57% of diagnosable cases out of 871 cases. For context, the current diagnostic rate for such disorders is 30-40%. With additional engineering features, AIM improved its accuracy from 54% to 66%, indicating AIM’s ability to capture underlying patterns within the additional training data. Like other algorithms, AIM performed worse for recessive cases, leading to the development of AIM-Recessive, achieving a 63.4% accuracy rate. Finally, a new AIM model without connections to established disease databases could potentially identify new disease genes and variants with limited patient data. Despite these successes, AIM was mainly trained on cases involving coding variants, with its ability to analyze non-coding variants being unclear.
In conclusion, the authors assessed AIM’s ability to diagnose Mendelian disorders by analyzing gene variants. Its superiority over existing algorithms demonstrated its potential to become a more cost-effective method of interpreting genetic variations and improving patient outcomes.
Image: PD
©2024 2 Minute Medicine, Inc. All rights reserved. No works may be reproduced without expressed written consent from 2 Minute Medicine, Inc. Inquire about licensing here. No article should be construed as medical advice and is not intended as such by the authors or by 2 Minute Medicine, Inc.
