liveBook · Manning

Cross-validation prediction

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A prediction produced for an observation by a model trained without the validation fold containing that observation.

Cross-validation prediction produces an out-of-fold prediction for every training case: each prediction comes from a model that did not train on that case’s validation fold. This provides unbiased case-level predictions for comparing with observed targets, rather than only an averaged cross-validation score. section 7.2.3

Use the predictions to:

  • Inspect performance across different subsets of the data. section 7.2.3
  • Identify possible overfitting or underfitting by comparing predictions with expected target values. section 7.2.3

Rental-price baseline

A linear-regression baseline uses sklearn.model_selection.cross_val_predict to estimate rental prices for all cases. The resulting predictions include implausible negative values, with R-squared 0.320, RMSE 17,197, and MAE 12,568 Yen, making the baseline’s weaknesses visible.

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Explains fold-based prediction and diagnostics, then stops short of deriving the cross-validation algorithm mathematically.
Examples
Demonstrates rental-price prediction with a linear baseline, negative outputs, diagnostic metrics, and coefficient inspection for unusual cases.
What it assumes
Assumes a prepared feature matrix, target values, a fitted model interface, and defined cross-validation splits.
What you can do after
Diagnose model behavior with sklearn.model_selection.cross_val_predict and compare predictions against targets using regression metrics.
How it connects
Connects cross-validation prediction with validation folds, evaluation metrics, linear regression, overfitting, and underfitting.
Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
liveBookmanning.com
Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.