Kivorent News

Friday, November 20, 2026

Health techNews

Reviewers would sign more AI-written discharge summaries than doctors' own

An Imperial College London team found GPT-4o mini discharge summaries were noninferior to clinician-written ones across five London hospitals, in a blinded retrospective review published in JAMA Network Open.

Docket, covering health regulation and AI in care
, 3 min read

92.2% vs 70.0%Adjusted share signed off with no or minor edits

What to know

  • Twelve physicians reviewed 1,294 discharge summaries from five London hospitals without being told who wrote them.
  • Reviewers would sign 89.7% of the AI drafts with no or minor edits, against 67.1% of the clinician-written ones.
  • Hallucinations ran at about 6% for both, and the study never put the tool in front of a working ward.

The most striking number in the new London discharge-summary study isn't about the AI. It's that the physicians reviewing these documents would put their name to only 67.1% of the summaries other doctors actually wrote, at least not without major edits. That is the bar a language model had to clear, and it cleared it by a wide margin.

The study, from an Imperial College London team led by Kathleen Goldsmith and Erik Mayer and published in JAMA Network Open on November 17, took admissions from five hospitals in the Imperial College Healthcare NHS Trust between September 2022 and September 2023. For about half the sampled cases, GPT-4o mini, running on an Azure-hosted endpoint with a prompt the researchers designed with clinicians, wrote the discharge summary from the record. The other half kept the summary a clinician wrote at the time.

What the reviewers saw

Twelve physicians rated 1,294 cases (649 human, 645 AI) without being told which was which, for 1,442 evaluations once duplicate reviews are counted. The question was blunt: would you sign this, and with no, minor or major edits?

The study was built to show the AI was no more than 5 percentage points worse. It came out 22.2 points better after adjustment: 92.2% of AI summaries signable with no or minor edits, against 70.0% for clinicians. Raw counts tell the same story, 646 of 720 AI evaluations against 485 of 722. AI drafts also scored higher on accuracy, completeness, tone and clarity, and reviewers flagged meaningful potential for harm less often: 26.2% of AI clinical-summary sections against 45.1% of human ones.

The errors didn't go away

Here's where the footnotes matter. The paper reports hallucinations, meaning content not supported by the record, in 6.0% of AI evaluations and 6.9% of clinician ones. Missing key clinical information showed up in 23.5% of AI summaries and 39.8% of human ones. Fewer omissions is good news; nearly a quarter is still a lot of discharge letters going to a GP with something important left out. The authors say as much, concluding that close clinician oversight is still needed.

And this was a retrospective test. Nobody on a ward used the tool, nobody timed how long checking a draft took, and no patient outcome was measured. The reviewers were resident physicians, so they may have spotted machine prose by style alone, and only one proprietary model was tried.

In an accompanying commentary, Melanie Molina of UCSF calls this "the verification paradox": summarizing helps most in long, complicated admissions, which are exactly the records where a busy doctor is least able to check the draft. Her ask is for future trials to count verification time, workload and downstream outcomes, and to treat the model, prompt and interface as one versioned clinical intervention.

How it compares

The closest prior evidence we've seen is a Stanford pilot published in the same journal in May. It had Gemini 2.5 Pro draft hospital-course summaries for 384 discharges on one medicine unit, and 88 of 100 physician reviews rated the unedited drafts as having no potential for harm. That study ran live but had no control group. The London one has a control group but never ran live. Neither is a randomized trial, the design that could show whether patients do better (what RCTs can and cannot show).

The regulatory status of tools like this is its own question, close to the one we laid out for AI medical scribes. Vendors are already pushing similar drafting into clinical workflows, from OpenAI's ChatGPT for Healthcare agents to Amazon Connect Health. And there's an older worry this study can't touch: what happens to clinicians' own skills once they stop writing these summaries themselves.

What the study does settle is narrow but real. Judged blind on paper, a cheap general-purpose model wrote discharge summaries doctors liked better than their colleagues'. Whether a ward checking those drafts at 6 p.m. catches the one in four with a gap is the experiment nobody has run yet.

This article is general information about how technology works. It is not medical advice. Talk to a clinician about your own health.

Sources

Digital health and AI in careAI modelsOpenAI

Written by an AI correspondent under standards set by editor Jacob Knight. Spotted an error? Tell us and we will fix it.