How deeply does Machine Learning System Design cover mean target encoding?

Defines regularized category-wise target means and flags leakage, but does not detail the encoding algorithm.

Mean target encoding

As explained in Machine Learning System Design

Target encoding is a categorical-variable encoding technique that replaces each category with a statistic of the target values associated with that category.

Mean target encoding replaces each category with the mean target value associated with that category, usually with some regularization. It offers an alternative to one-hot, ordinal, and hashing-based encodings for categorical features. section 11.1.2 section 11.1.3

Use it with an explicit leakage check:

Treat the encoded category statistic as a feature for the downstream model. section 11.1.3
Prevent the encoding from using target information unavailable during inference; otherwise, overfitting and poor new-data performance can result. section 11.1.2 section 11.1.3
Use advanced validation techniques such as nested cross-validation when evaluating the feature. section 11.1.3

Categorical features in ETA prediction

An ETA system can combine continuous, categorical, and geospatial inputs. Categorical inputs may be transformed through mean target encoding or other categorical encodings before entering a machine-learning model; DeepETA illustrates a broader pipeline that converts diverse tabular inputs into discrete tokens and learnable embeddings. section 11.1.2

How Machine Learning System Design covers it

liveBook manning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Ensemble Methods for Machine LearningGreedy target encoding uses target-derived statistics computed from training data to encode categorical values, potentially introducing target leakage when the encoded examples contribute to those statistics.
liveBook manning.com

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA feature encoding technique that replaces each category with a statistic computed from the target values associated with that category.

Compare how 3 books explain mean target encoding →

Related concepts

TargetEncoderimplemented by · TargetEncoder is an encoding method that replaces categorical values with statistics computed from the target variable, often using smoothing.
target leakingproduces · Target leakage is the inclusion or use of information derived from the prediction target that would be unavailable when making predictions.
supervised encodingkind of · Supervised encoding is a categorical-feature encoding technique that uses target values when deriving category representations.
labeluses · A label is a value or annotation associated with an item that identifies its target category, outcome, or reference answer for learning or evaluation.
embeddingalternative to · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
high-cardinality categorical featureaddresses · A high-cardinality categorical feature is a categorical feature with many distinct values relative to its dataset or analysis context.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.