Machine learning model demonstrates superior screening efficiency for Hepatitis C virus carriers
1. Dagan and colleagues developed machine learning (ML) models using retrospective patient data to identify potential hepatitis C virus (HCV) carriers.
2. The top-performing ML model demonstrated over 100-fold greater efficiency at identifying HCV carriers than the traditional screening approach.
Evidence Rating Level: 2 (Good)
Study Rundown: Disease screening methods using broad screening criteria have high sensitivity rates but low adherence among high-risk individuals. Machine learning (ML) strategies have the potential to identify high-risk individuals more efficiently using information from the electronic health record (EHR). Dagan and colleagues trained multiple ML models that used relevant predictors, such as sociodemographic data, health behaviors, and past diagnoses, to discriminate patients into high-risk or low-risk groups. The best-performing model – XG Boost, was used for the prospective evaluation and was compared to the standard US Preventive Services Task Force (USPSTF) screening guidelines. XGBoost identified 677 patients in the top 0.1% risk group, and 164 agreed to screening, with a seropositive rate of 20.1%. Among the 164, 76.9% had an active infection, which extrapolated to a number needed to screen (NNS) of 6.5 patients. Meanwhile, the USPSTF approach screened 53,403 individuals and only achieved a seropositivity of 0.5%, and 19.6% had active infection. This data was extrapolated to an NNS of 1029. Overall, this study demonstrated a much higher screening efficiency when using an ML model to identify high-risk patients than traditional screening criteria.
Click here to read the study in NEJM AI
Relevant Reading: Machine learning for prediction of viral hepatitis: A systematic review and meta-analysis
In-Depth [prospective cohort]: 492,290 patients between the ages of 18-79 without prior HCV tests, diagnoses, or medications were included as the retrospective cohort to develop the ML models. Predictors from EHR, such as existing risk factors, sociodemographic data, health behaviors, and previous diagnoses, were used to train three ML models. Discrimination was assessed using the area under the receiver operating characteristics (AUROC) curve and sensitivity. XGBoost demonstrated the highest discrimination ability, with an AUROC of 0.95 (0.93-0.96, 95% CI) and a sensitivity of 67.6% when screening the top 5% riskiest patients. Then, XGBoost was used to identify prospective high-risk patients for screening. Its extrapolated positive predictive value (PPV) and NNS (calculated as 1 / PPV) were compared to the standard USPSTF screening criteria. XGBoost identified 677 individuals in the top 0.1% risk group, 768 in the 0.1-0.2% risk group, and 781 in the 0.2-0.3% risk group. The top 0.1% risk group achieved a PPV of 15.5 (NNS = 6.5), as compared to a PPV of 6.9 (NNS = 14.5) in the 0.1-0.2% risk group, and a PPV of 6.3 (NNS = 15.9) in the 0.2-0.3% risk group. All these high-risk groups identified by XGBoost demonstrated a screening efficiency 64-158 times higher than the standard USPSTF group, which only had a PPV of 0.1 and an NNS of 1,028.8. One limitation was the potential depletion of the highest-risk patients in the USPSTF group, as many were likely identified by and screened as part of the XGBoost groups. In summary, this study provided evidence supporting the use of ML-driven screening methods based on EHR data and could potentially reduce the burden on primary care clinicians and improve their ability to engage in preventive medicine.
Image: PD
©2024 2 Minute Medicine, Inc. All rights reserved. No works may be reproduced without expressed written consent from 2 Minute Medicine, Inc. Inquire about licensing here. No article should be construed as medical advice and is not intended as such by the authors or by 2 Minute Medicine, Inc.
