Grouped cross-validation

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Group cross-validation partitions data so that all observations belonging to a group remain in either training or validation folds.

When related rows belong to the same group, keep every member in either training or validation folds. This prevents group relationships from distorting model evaluation, especially with non-IID tabular data. Prefer grouped cross-validation when hidden groups exist; use time-based validation instead when time determines the model. section 2.1.1

Figure 2.1 Comparison of IID and non--IID data sequences. The left chart shows IID data, while the right chart illustrates non-IID data, highlighting the sequential correlation patterns.

Compare IID and non-IID row sequences to see why related observations can require grouping during validation. section 2.1.1

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes

Defines the fold-partitioning rule and recommendation but stops short of implementation steps or algorithmic internals.

Examples

Uses hidden groups in tabular data and contrasts grouping with time-based validation when time determines predictions.

What you can do after

Apply grouped folds to keep related observations together during model validation.

How it connects

Connects grouped cross-validation with non-IID data, time-based validation, time features, and training-validation-testing splits.

liveBookmanning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Related concepts

• cross-validationkind of · Cross-validation is a resampling technique that evaluates a model by repeatedly fitting it on some portions of available data and assessing it on held-out portions.
• data leakageaddresses · Data leakage is the unintended exposure or use of information beyond its intended boundary, compromising confidentiality or the validity of model development and evaluation.
• validationused by · The process of checking data, behavior, systems, models, or results against defined criteria to assess their correctness, suitability, or acceptability.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.