Supervised learning learns from historical input–output pairs to predict an outcome for a new input. Its central question is predictive: given x, what y is likely? Typical uses include forecasting sales, identifying spam, and prioritizing patients by their expected need for intensive care. section 4.0 section 4.1.1
Given data D = {(xᵢ, yᵢ)}, it selects a predictive function f so that f(x) estimates y. Inputs may be images, patient characteristics, or sales so far. When y is categorical, the task is classification; when y is continuous, it is regression. The model family can be linear models, decision trees, k-nearest neighbors, or deep learning models. section 4.1
Data fitting means choosing a function that predicts past observations well while aiming to generalize. Under square loss, the best prediction for inputs x₁, …, xₘ is the conditional expectation E[Y|x₁, …, xₘ]. For binary classification, this expectation equals P(Y = 1|x₁, …, xₘ). section 4.1.2
Prediction is not an intervention
A hospital can use age and existing health conditions to predict how soon a COVID-19 patient may need ICU care, then prioritize patients accordingly. That useful prediction does not answer which factors would reduce ICU admissions if changed. The same variables and model form can serve causal analysis, but causal analysis asks about the effects of actions rather than prediction accuracy alone. section 4.1.1
Figure 4.1 When to use causal inference or machine learning. Model input is denoted by X and the outcome by Y.
Use the decision diagram to distinguish a model used to forecast an observed outcome from one used to decide what changing an input would cause. section 4.1.1
Reliable predictive use depends on limits that accuracy alone cannot settle:
- Expect future cases to follow the same underlying dynamics as historical cases; the usual formal assumption treats past and future input–outcome pairs as independent and identically distributed. section 4.1.3
- Be cautious when decisions based on predictions change the system itself. Interventions can shift the outcome distribution and make an originally useful predictor inaccurate. section 4.1.3
- Check for spurious correlations. A dog-versus-fish classifier trained on dogs on grass and fish in water may learn background color rather than animal features, then fail on a dog against a blue background. section 4.3.3
Supervised learning can also support causal inference without becoming causal inference itself. In an adjustment formula, it can estimate conditional probabilities such as P(Y = y|X = x, Z = z), especially when many confounders make direct counting sparse. This plug-in use still requires causal assumptions about confounding; cross-validation alone is not enough to choose a causal model. section 4.2 section 4.6
Avoid training a flexible model and predicting on the same observations when its predictions will feed causal adjustment. The fitted model can mimic observed outcomes rather than the conditional expectation. Cross-fitting separates training and prediction folds to reduce this overfitting risk. section 4.2.5
