liveBook · Manning

Cross-validation

As explained in Causal AI

Cross-validation is a model evaluation technique that assesses predictive performance on data withheld from training.

Cross-validation evaluates how well a model predicts data withheld from training. It therefore answers a predictive question rather than merely asking how closely the model fits its training data. section 2.3.4

Use cross-validation to distinguish predictive performance from training-data fit:

  • Goodness-of-fit measures how well a model represents the data used for training. section 2.3.4
  • Cross-validation measures prediction on data the model did not train on, which better matches machine-learning prediction goals. section 2.3.4
  • Strong held-out prediction does not establish that the model’s causal inferences are correct. section 2.3.4

Fit versus generalization

A model can have better goodness-of-fit than competing models while predicting withheld data poorly; cross-validation exposes that difference. section 2.3.4

How Causal AI covers it

How deep it goes
Defines held-out prediction and contrasts it with goodness-of-fit, but does not specify fold construction or computational procedure.
Examples
Uses a model with strong training-data fit but poor withheld-data prediction as the central comparison.
What it assumes
Assumes familiarity with trained models and goodness-of-fit while clarifying why prediction and causal inference differ.
What you can do after
Evaluate models by distinguishing generalization to unseen data from fit to training data.
How it connects
Connects predictive modeling with goodness-of-fit statistics and causal inference.
Cover of Causal AI
liveBookmanning.com
Read Causal AI on liveBook Get the book at manning.com

Also explained in

Compare how 12 books explain cross-validation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Causal AI. The section links point to the parts of the book it draws on.