Medical chatbots pass the exams. What would it take to know they help patients?

In July 2023, a few months after ChatGPT was released, Nature Medicine published a review titled "Large language models in medicine". It has since become one of the most cited introductions to the subject for clinicians. Its first author, Arun Thirunavukarasu, was then a medical student at Cambridge. He is now an academic ophthalmology trainee in London and an honorary research fellow at the London School of Hygiene and Tropical Medicine.

The review described the performance of these systems in medicine as "impressive but mixed". Asked about that phrase in an interview for Tech Anatomy in October 2026, three years and several model generations later, Thirunavukarasu located the mixed part in one specific trait: a large language model, or LLM, is "equally fluent whether or not what it's telling you is correct". An LLM is a system trained on very large amounts of human-written text and then tuned to answer questions in conversation. Its fluency is what makes it convincing, and it does not drop when the answer is wrong.

What the exam results measured

Much of the early research tested LLMs the way medicine tests its own trainees: with written examinations. Text-based exams were the most convenient benchmark for systems that work in text. In a study published in PLOS Digital Health in April 2024, Thirunavukarasu and colleagues put several models through questions from the Part 2 examination of the UK Royal College of Ophthalmologists, one of the hurdles before practising independently as an eye surgeon. On an 87-question mock exam, GPT-4 answered 69 percent correctly. Expert ophthalmologists in the same study had a median score of 76 percent, ophthalmology trainees 59 percent, and junior doctors outside the specialty 43 percent.

The authors themselves flagged a limit: some questions might have appeared in the models' training data, which would inflate scores through recall rather than reasoning.

Asked whether passing such an exam puts a model on a par with a specialist, Thirunavukarasu answers no. The exam is one hurdle among many in a surgeon's training, and many clinical skills are never examined in writing. In his framing, an exam pass may be a necessary condition for a model to play any role in care, but it is not a sufficient one to trust it without oversight.

Why the stronger evidence is scarce

The kind of study that would show an effect on patients is, in his view, pragmatic and randomised. Pragmatic means run in ordinary clinical conditions, with messy histories and patients who describe symptoms more or less clearly. Randomised means that physicians, hospitals or patients are allocated by chance to care with or without the tool, so that differences in outcomes can be attributed to it.

Such trials are rare, and he gives three reasons that do not involve bad faith. They are slow and expensive. Testing a model in real care raises a genuine safety question: if the tool is used without adequate oversight, trial participants could be harmed. And the technology moves faster than the trial. A company that funds a study of a fixed application knows a newer model may supersede it within months. His proposal is to lock an application down, describe exactly how it works, and test that version anyway, accepting that the result will describe a snapshot.

He also points out that this is not specific to AI. Most medical research on any question is not randomised, because other designs are cheaper, faster and easier to run.

A reporting standard for chatbot studies

Thirunavukarasu is one of the authors of CHART, a reporting guideline for studies that evaluate chatbots giving health advice, published in August 2025. Many of its items are standard for any research, such as stating the analysis plan in advance. Others are specific to generative AI: naming the exact model, whether it was modified or fine-tuned, and when it was queried. The last point matters because commercial models are updated behind the scenes, often without notice, so the same name can refer to different systems from one month to the next.

The guideline responds to a pattern he describes in the existing literature: studies that report too little about what was tested, and an incentive to publish striking results, whether a model does very well or very badly.

Two readings of "these models are already seeing patients"

When asked whether a model that passes medical boards is ready to see patients, Thirunavukarasu calls the statement false, then adds a qualification: patients already ask chatbots for medical advice, the way they once searched Google, so "whether or not we like it, these models are seeing patients".

That sentence can support two different conclusions. One reading is that use has already outrun evidence, so the priority is to slow deployment until pragmatic trials catch up. The other is that, since use is happening anyway, the more useful work is to study it where it occurs, set reporting standards, and teach clinicians and patients to read the outputs critically. Thirunavukarasu's own position leans towards the second, but he presents it as a research agenda, not as a settled answer.

The same tension runs through his work on training. He and colleagues use the term "never-skilling" for trainees who delegate so much reasoning to AI that they never acquire certain skills in the first place, as distinct from deskilling, the loss of skills already learned. He is careful to say this may or may not be a problem: if the work changes, some skills may no longer be needed. He also sees an opening. If models can handle recall, medical education could spend more time on statistical and Bayesian reasoning and on appraising research, two areas he considers poorly taught today.

What the interview does not establish

The interview reflects one researcher's view, informed by his own studies, and it is not a review of the field. The exam figures above come from one specialty and from models available in 2023, and Thirunavukarasu notes that newer models have since done better, in some cases outscoring experts. Results on medical images were weaker: in a study he took part in, general-purpose LLMs did less well on ophthalmic images than specialised deep learning models trained on those images. That gap says more about how each model was trained than about "AI" in general, a distinction he insists on throughout the conversation.

What to watch

The exam studies show that models can answer many medical questions. Whether their use changes outcomes for patients is a separate question, and it is still open. The indicators worth following are the randomised trials he says are running in real clinical settings, whether published chatbot studies start to report the details CHART asks for, and whether regulators and professional bodies define the clinical tasks a model must perform before it is trusted with them. Thirunavukarasu suggests borrowing from autonomous driving, where the task was broken down into separate behaviours, each tested on its own. Whether a clinical consultation can be broken down and tested in the same way has not yet been shown.

Listen to the episode

These models are already seeing patients - Dr Arun Thirunavukarasu, LLMs in Medicine

Listen · 34 min