Generative artificial intelligence comparable in terms of clinical accuracy and report quality for chest radiographs

1. Generative artificial intelligence (AI) and radiologist reports received comparable ratings of clinical accuracy and report quality, that were significantly greater than teleradiology report ratings. 

Evidence Rating Level: 1 (Excellent)

Timely diagnostic imaging interpretation in the ED is crucial for patient care. Immediate radiologist interpretation minimizes errors, but some EDs lack consistent radiology services. Generative AI methods may bridge this gap by offering near-instant, fatigue-free interpretations of medical imaging. This study aimed to develop a generative AI tool for chest radiograph interpretation and evaluate its performance in the ED. The study data set comprised 500 randomly sampled chest radiographs of patients (mean [SD] age 53.3 [21.6] years, 56.4% female, 32.8% abnormal radiographs) evaluated in an ED. Radiologist, teleradiologist, and AI-generated reports for each radiograph were deidentified and rated on a Likert scale by 6 practicing board-certified emergency medicine physicians for report quality and clinical accuracy. AI and radiologist reports received significantly higher ratings than teleradiology (AI: mean [SE] 3.22 [0.34], P < .001; radiologist: 3.34 [0.34], P < .001; teleradiology: 2.74 [0.34]). AI and radiologist report ratings were not significantly different. There was no significant difference in the probability that each report type would generate a non-clinically discrepant report for normal and abnormal studies, even after stratifying by the presence of cardiomegaly, pulmonary edema, pleural effusion, infiltrate, pneumothorax, and support devices. When using radiologist reports as the standard, the sensitivity and specificity of AI reports for detecting any abnormality were 84.8% and 98.5%, respectively, and 91.5% and 97.0% for teleradiology reports. Overall, these findings suggest that the generative AI model produced reports similar in performance to a radiologist and better than a teleradiology service in a representative sample of ED chest radiographs. 

Click to read the study in JAMA Network Open

Image: PD

©2023 2 Minute Medicine, Inc. All rights reserved. No works may be reproduced without expressed written consent from 2 Minute Medicine, Inc. Inquire about licensing here. No article should be construed as medical advice and is not intended as such by the authors or by 2 Minute Medicine, Inc.