Mean target encoding

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover mean target encoding?

Defines regression and classification statistics, explains smoothing and overfitting risk, and demonstrates pipeline integration without deriving the estimator.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A feature encoding technique that replaces each category with a statistic computed from the target values associated with that category.

Mean target encoding converts a high-cardinality categorical feature into numeric values that summarize the target associated with each category. It is useful when IDs, zip codes, product names, or geographical names would create too many one-hot features for practical modeling. section 2.2.5 section 6.1.3

For regression, each category receives its average target value. For classification, it receives a conditional probability or odds ratio. Because rare categories can overfit, smoothing blends the category estimate with the overall training-set target average. section 6.1.3 section 6.1.3

A practical workflow is:

Identify categorical features whose cardinality is too large for convenient one-hot encoding; no fixed threshold applies across datasets. section 6.1.3
Apply target encoding to the selected columns while retaining other preprocessing choices for lower-cardinality categories. section 6.0
Choose a smoothing value above zero to balance category-specific target information against the global target average, then tune it for the dataset. section 6.1.3
Prefer target encoding especially with gradient boosting; use embeddings instead when a deep-learning approach is the better fit. section 2.2.5

Pipeline application

The category_encoders TargetEncoder accepts selected high-cardinality columns and a smoothing value, and can be combined with XGBoost and cross-validation in a scikit-learn-compatible pipeline. section 6.1.3

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Ensemble Methods for Machine LearningGreedy target encoding uses target-derived statistics computed from training data to encode categorical values, potentially introducing target leakage when the encoded examples contribute to those statistics.
liveBook manning.com

Machine Learning System DesignTarget encoding is a categorical-variable encoding technique that replaces each category with a statistic of the target values associated with that category.

Compare how 3 books explain mean target encoding →

Related concepts

TargetEncoderimplemented by · TargetEncoder is an encoding method that replaces categorical values with statistics computed from the target variable, often using smoothing.
target leakingproduces · Target leakage is the inclusion or use of information derived from the prediction target that would be unavailable when making predictions.
supervised encodingkind of · Supervised encoding is a categorical-feature encoding technique that uses target values when deriving category representations.
labeluses · A label is a value or annotation associated with an item that identifies its target category, outcome, or reference answer for learning or evaluation.
embeddingalternative to · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
high-cardinality categorical featureaddresses · A high-cardinality categorical feature is a categorical feature with many distinct values relative to its dataset or analysis context.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.