AI in medical imaging, ten years after "stop training radiologists"

In 2016, Geoffrey Hinton, one of the researchers behind modern deep learning and later a winner of the Turing Award and the Nobel Prize in Physics, told a conference audience: "People should stop training radiologists now. It's just completely obvious that within five years, deep learning is going to do better than radiologists."

The prediction was an extrapolation from neural networks beating benchmarks that had resisted every earlier method. Ten years later, the US Food and Drug Administration says it has authorised more than 1,600 AI-enabled medical devices, the majority of them in radiology, and radiology departments are still hiring. At the Mayo Clinic, the radiology staff has grown by 55 percent since 2016, to more than 400 radiologists, according to The New York Times. A Tech Anatomy ERROR 404 episode retraces how both became true at once.

Before deep learning: computer-aided detection

Software that reads medical images is older than deep learning. From the late 1980s, researchers at the Kurt Rossmann Laboratories of the University of Chicago built computer-aided detection, or CAD, systems for mammography. These relied on rules written by engineers, such as marking clusters of bright pixels. They did not diagnose; they underlined regions worth a second look, a little like a spellchecker.

In June 1998, the FDA approved the ImageChecker M1000 from R2 Technology, a company founded to commercialise this work. CAD became widespread in American screening, and its limits showed up later. A study published in JAMA Internal Medicine in 2015, covering more than 625,000 digital screening mammograms read between 2003 and 2009, found that CAD did not improve sensitivity, specificity or cancer detection.

From ImageNet to the first authorisations

The method that changed the field came from outside medicine. In 2012, AlexNet, a deep neural network from Hinton's group, won the ImageNet image recognition challenge by a wide margin. A convolutional neural network learns visual features in layers, edges first, then shapes, then structures, from examples rather than hand-written rules. Imaging researchers wondered what such a network could learn from millions of scans.

Results followed quickly. In December 2016, a Google team reported in JAMA a deep learning algorithm that detected referable diabetic retinopathy in retinal photographs with sensitivity above 90 percent. In early 2017, a Stanford team showed in Nature that a network trained on 129,450 clinical images classified skin lesions at a level comparable to 21 board-certified dermatologists.

Regulatory decisions came in 2018. In February, the FDA granted a De Novo authorisation, the pathway for new types of low to moderate risk devices, to Viz.ai's software that flags suspected large vessel occlusion strokes on CT scans and alerts a specialist. In a retrospective study of 300 scans submitted by the company, the notification arrived earlier than under standard care in more than 95 percent of cases, by 52 minutes on average. In April, IDx-DR became the first device authorised to give a screening result for diabetic retinopathy without a clinician interpreting the image. Its pivotal trial in primary care reported 87.2 percent sensitivity and 90.7 percent specificity.

Why accuracy in the lab did not travel

The episode's central argument is that matching a specialist on a test set is where the difficult part begins.

A review of COVID-19 imaging models published in Nature Machine Intelligence in 2021 screened 2,212 studies, retained 62 after quality assessment, and concluded that none of the models was of potential clinical use, mainly because of small or biased datasets and missing external validation. A systematic review in Radiology: Artificial Intelligence in 2022 found that 70 of 86 deep learning algorithms, or 81 percent, performed worse when tested on data from another institution, and nearly a quarter performed substantially worse. Scanners, protocols and populations differ between hospitals.

Bias is a related problem. A 2021 study in Nature Medicine found that chest X-ray classifiers underdiagnosed disease more often in under-served groups, including Black patients. A 2022 study in The Lancet Digital Health showed that models can predict self-reported race from images alone, which radiologists cannot do.

The human side matters too. In a 2023 experiment published in Radiology, 27 radiologists read mammograms accompanied by AI suggestions, some deliberately wrong. Very experienced readers rated 82.3 percent of cases correctly when the suggestion was right and 45.5 percent when it was wrong. Less experienced readers were affected more. This automation bias is familiar from aviation.

What the largest screening study measured

The strongest evidence so far comes from Germany. The PRAIM study, published in Nature Medicine in 2025, followed 463,094 women screened between July 2021 and February 2023 at 12 sites by 119 radiologists, who chose case by case whether to use an AI system called Vara MG. With AI support, the cancer detection rate was 6.7 per 1,000 women against 5.7 per 1,000 without, a relative increase of 17.6 percent, and the recall rate was not higher.

Hinton revised his view in 2025. In an email to The New York Times, he said he had spoken too broadly, that he had meant image analysis only, and that he had been wrong on timing but not on direction. He now expects most image interpretation to be done by "a combination of AI and a radiologist". Curtis Langlotz of Stanford had made a similar case in a 2019 editorial: "Radiologists who use AI will replace radiologists who don't."

Two readings of the record

One reading treats the past decade as a normal adoption curve: the first authorisations date from 2018, the evidence has moved from retrospective benchmarks to population-scale prospective studies, and PRAIM shows a measurable benefit in routine screening. On this view, the gap between authorisation and use is a matter of integration, payment and training, and it is closing.

A second reading is more cautious. CAD was also adopted widely before large studies found it added little, and most AI-enabled devices reach the market through pathways that do not require trials showing a change in patient outcomes. On this view, the number of authorised devices measures regulatory activity, not clinical value, and each tool still needs local evidence before it earns trust.

What the evidence does not show

PRAIM was observational, not randomised. Radiologists decided when to use the AI, so the groups may differ in ways the analysis cannot fully remove, and the study was funded by the software's developer, according to its disclosures. A higher detection rate is also not yet a lower death rate: some additional cancers may be small tumours that would never have caused harm, and the authors note that interval cancers and stage at diagnosis are still being followed. The FDA's list counts authorisations, not use in daily practice.

What to watch

The indicators that would separate the two readings are follow-up data from screening programmes using AI, showing whether interval cancers fall; randomised screening trials that report interval cancer rates; independent validation of commercial tools at hospitals other than the ones that trained them; and whether foundation models, which combine images with clinical notes, lab results and prior scans, perform as well outside their development sites as inside them. None of them is settled yet.

Listen to the episode

ERROR 404 - The Story of AI in Medical Imaging

Listen · 21 min