Large language models (LLMs) substantially improved patient comprehension of discharge summaries

1. Kumar and colleagues assessed LLM’s ability to translate discharge summary notes into plain language to improve patient comprehension.

2. Patients, especially those with limited health literacy, reported greater subjective and objective comprehension of the translated clinical notes.

Evidence Rating Level: 3 (Average)

Study Rundown: Patients often have access to their clinical notes, but many have difficulty understanding and utilizing them for health decisions. Kumar and colleagues explored the use of LLMs, such as Generative Pretrained Transformer 4 (GPT-4), to translate discharge summary notes (DSNs) into plain language. 553 adult participants who could read in English and had no cognitive impairment completed the study. They were instructed to read four DSNs related to four common primary admission diagnoses: congestive heart failure (CHF), community-acquired pneumonia (CAP), diabetic ketoacidosis (DKA), and acute ischemic stroke (AIS). Each participant was randomly given two untranslated DSN and two translated DSN. After reading the DSNs, patients completed surveys that assessed subjective comprehension and confidence level and multiple-choice questions that assessed objective comprehension. The study found that DSNs translated by GPT-4 significantly improved subjective and objective comprehension, and self-reported confidence scores across all four diagnoses. This study demonstrated that GPT-4-based translation substantially improved patient comprehension of clinical documentation.

Click here to read the study in NEJM AI

Relevant Reading: The Impact of Artificial Intelligence on Medical Professional Language and Doctor-Patient Interactions

In-Depth [cross-sectional study]: The most representative DSN for the four diagnoses and the prompts that generated the most readable and comprehensive translations in GPT-4 were selected by the research team. Then, 1039 adult participants who could read in English and had no cognitive impairment were enrolled in the study. Their baseline information (e.g., age, gender, race/ethnicity, education level, self-reported health literacy level) was collected. Each participant was randomly given two untranslated DSN and two translated DSN. After reading the DSNs, patients completed surveys that assessed subjective comprehension and confidence levels on five-point and three-point Likert scales, respectively. Objective comprehension was evaluated using four multiple-choice questions. Out of the 1039 enrolled participants, 553 completed the study. The participants who did not complete the study were disproportionately Black and had an education level below a Bachelor’s degree. Across all four diagnoses, subjective comprehension scores increased by 2.4 points (95% confidence interval [CI], 2.15-2.71, p < 0.001). Objective comprehension scores increased by 1.2 points (95% CI, 1.09-1.26, p < 0.001), and confidence scores increased by 2.0 points (95% CI, 1.82-2.13, p < 0.001). Additionally, the effect of GPT-4-based translations on patient comprehension scores was substantially greater in Black and Hispanic populations and patients with lower reported levels of health knowledge. Overall, this study provides promising evidence that GPT-4 can be used to improve patients’ understanding of the content in clinical documentation.

Image: PD

©2026 2 Minute Medicine, Inc. All rights reserved. No works may be reproduced without expressed written consent from 2 Minute Medicine, Inc. Inquire about licensing here. No article should be construed as medical advice and is not intended as such by the authors or by 2 Minute Medicine, Inc.