Machine learning in five steps, and where each one can go wrong in healthcare

Ask a hospital data team how a prediction model was built and the answer usually follows the same five steps. Collect examples where the right answer is already known. Clean them. Choose a mathematical function. Adjust that function until its predictions match the known answers as closely as possible. Then check it on cases it has never seen. This is supervised learning, the branch of machine learning behind most clinical prediction tools, and it was the subject of a two-part Tech Anatomy explainer in October 2025.

The idea at its core is modest. Instead of writing rules ("if blood sugar is above X, flag the patient"), the developer gives the computer a table in which each row is a patient, each column a measurement, and a final column the outcome. The computer looks for the relationship between the columns and the outcome. Nothing in this is magic; it is statistics applied at scale. The steps are simple to list. Where the difficulty sits, and where models fail, depends on the step.

Three tasks, one sequence

The second episode walked through three classic hospital tasks. Each uses the same sequence, with a different kind of answer at the end.

The first is telling benign tumours from malignant ones. A well-known public example is the Wisconsin Diagnostic Breast Cancer dataset, assembled at the University of Wisconsin in the 1990s and still hosted by the UCI Machine Learning Repository. It describes 569 breast masses through 30 measurements of cell nuclei taken from digitised images of fine needle aspirates, each labelled benign or malignant. Because the answer has two possible values, a model such as logistic regression, which outputs a probability between 0 and 1, is a natural first choice.

The second is predicting which patients will be readmitted within 30 days of discharge. In the United States the question carries financial weight: since October 2012, the Hospital Readmissions Reduction Program has cut Medicare payments to hospitals with excess readmissions for certain conditions. One widely cited tool, the LACE index, published in the Canadian Medical Association Journal in 2010, scores four items (length of stay, acuity of admission, comorbidity and emergency visits in the previous six months) to estimate the risk of death or unplanned readmission within 30 days. It was derived from 4,812 patients in 11 Ontario hospitals and then validated on administrative data for a random sample of one million Ontarians discharged between 2004 and 2008.

The third is estimating length of stay. Here the answer is a number of days rather than a category, so the episode used linear regression, the straight line of school mathematics, written y = wx + b, in which training means finding the values of w and b that best fit the data.

What "training" actually changes

Training sounds like something the machine undergoes. In practice it is a loop. The model starts with arbitrary values, makes predictions on the training data, measures how far each prediction is from the known answer (the error), and adjusts its parameters slightly to reduce that error. The episode compared this to turning the dial of a radio until the signal is clear. The loop runs thousands or millions of times. What the model ends up with is not a memory of the patients it saw but a set of numbers that summarise the relationship between inputs and outcome.

Testing is what tells the two apart. A model evaluated on its own training data can look excellent simply because it has fitted the quirks of those particular patients, a failure called overfitting. Hence the rule of keeping a separate test set, data the model never saw during training, and in medicine, increasingly, of testing on patients from other hospitals or other years. The LACE authors, for instance, measured discrimination with a C statistic of 0.684, meaning the index ranks a randomly chosen patient who died or was readmitted above one who was not in about 68% of pairs.

The steps that receive less attention

The episode described data cleaning as the step that is "often much harder than it sounds". Survey data points the same way. In Anaconda's 2020 State of Data Science survey of about 2,360 respondents, respondents reported spending about 45% of their working time loading and cleaning data, according to coverage of the results. Later editions of the same survey reported somewhat lower shares.

A second weak point sits in step one, in the choice of the outcome column. A model learns whatever the label measures, which is not always what its users think it measures. In 2019, Ziad Obermeyer and colleagues reported in Science on a commercial algorithm used by US health systems to select patients for extra care. It predicted future healthcare costs as a stand-in for health needs. Because less money was spent on Black patients at the same level of illness, the model systematically underrated their needs. Remedying this disparity, the authors estimated, would raise the share of Black patients receiving additional help from 17.7% to 46.5%. The authors located the bias in the choice of label rather than in the fitting itself: by some measures of predictive accuracy, cost looked like an effective proxy for health.

Two ways to read the simplicity

One reading takes the five steps as reassurance. If supervised learning is fitting a function to labelled examples, clinicians and managers can question a model the way they would question any study: who was in the data, what was the outcome, how was it tested. The explainer was built on this premise, that the jargon hides a familiar logic.

Another reading holds that the simplicity of the recipe hides where the judgement lies. Choosing a label, deciding which records to drop during cleaning, picking a test population: each is a decision with clinical and ethical consequences, and none is visible in the final accuracy figure. On this view, understanding the five steps is necessary but does not by itself let a reader evaluate a model.

What the explainer does not cover

The episodes deliberately kept to supervised learning on tables of numbers. Image models, language models and unsupervised methods, which find structure without a labelled answer, follow different routines. Good performance on a test set also says nothing about whether using the model changes care. The LACE study, for example, measured how well the index predicts death or readmission; it was not designed to show whether using the score in practice reduces either.

What to watch

For any clinical model, the questions follow the five steps: what the label measures, how much data was discarded and why, and whether the test population resembles the one where the tool will be used. The open question is whether hospitals will start publishing those answers alongside accuracy figures, and whether buyers will ask for them.

Listen to the episode

Machine learning made ridiculously simple (part 1)

Listen · 9 min