Cross-validation

As explained in Grokking Machine Learning

A resampling technique that partitions a dataset into k portions and repeatedly trains on k−1 portions while validating on the remaining portion, then averages the validation scores.

Have a small dataset and want to evaluate a model without setting aside one fixed validation portion? K-fold cross-validation rotates the data through training and validation. Split it into k roughly equal portions; train k times, holding out a different portion for validation each time; then average the validation scores. section 13.6

Four folds

Divide the data into four portions. Train four times, using three portions for training and the remaining portion for validation each time. The model’s final score is the average of the four validation scores. section 13.6

Figure 13.2 K-fold cross validation is a useful method that recycles our data to use it both as training and as validation. On the top, we see the classical training-validation split. On the bottom, we see an illustration of fourfold cross-validation in which we split the data into four equal (or almost equal) portions. We then train our model four times. Each time we pick three of the portions as the training set, and the remaining one as the validation set. The score of this model is the average of the four scores obtained on each of the validation sets.

Compare a conventional training-validation split with the four-fold rotation of training and validation portions. section 13.6

K-fold cross-validation is used in GridSearchCV; its results can be inspected through cv_results_. section 13.6

How Grokking Machine Learning covers it

How deep it goes

Splits data into k portions, trains k times with one portion held out each time, and averages validation scores.

Examples

Illustrates fourfold cross-validation by rotating the held-out validation portion across four training runs.

What you can do after

Evaluate a model by averaging validation scores from runs that each hold out a different data portion.

How it connects

Connects model evaluation with GridSearchCV and the training-validation-testing split.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningK-fold cross-validation partitions data into k folds and repeatedly trains and evaluates a model using one fold for validation and the others for training.
• 

  liveBookmanning.com
  Causal Inference for Data ScienceCross-validation is a model-evaluation procedure that repeatedly partitions data into training and validation portions to estimate predictive performance.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AICross-validation is a model-evaluation technique that repeatedly partitions data into training and validation subsets.
• 

  liveBookmanning.com
  Effective Conversational AIK-fold cross-validation is a model-evaluation technique that partitions a dataset into k folds and repeatedly evaluates a model with each fold held out in turn.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionK-fold cross-validation is an evaluation procedure that partitions data into K subsets and repeatedly trains on K−1 subsets while evaluating on the remaining subset.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA validation procedure that partitions data into K folds, trains on K−1 folds, and evaluates on the remaining fold in rotation.
• 

  liveBookmanning.com
  Grokking StatisticsCross-validation repeatedly partitions data into complementary training and evaluation folds to assess model performance.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentCross-validation is a resampling technique that repeatedly trains and evaluates a model on different partitions of available data.
• 

  liveBookmanning.com
  Causal AICross-validation is a model evaluation technique that assesses predictive performance on data withheld from training.
• 

  liveBookmanning.com
  Building LLM Applications with DSPyCross-validation is an evaluation technique that repeatedly partitions data into training and validation subsets to produce a more reliable estimate.
• 

  liveBookmanning.com
  Financial AI in PracticeCross-validation is a model evaluation technique that partitions data into subsets and repeatedly trains and evaluates a model on different subsets.

Compare how 12 books explain cross-validation →

Related concepts

• covariateuses · A covariate is an observed or supplied variable included in a statistical, causal, or predictive model to describe, explain, predict, or adjust for variation in an outcome.
• grid searchused by · Grid search is a search method that evaluates configurations formed from specified candidate values, typically every combination in their Cartesian product.
• training datasetuses · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.