liveBook · Manning

TargetEncoder

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

TargetEncoder is an encoder implementation that replaces categories with smoothed target-derived statistics.

TargetEncoder replaces each category with a smoothed target-derived statistic. Regression uses the category's average target; classification uses conditional probability or odds ratio. Smoothing blends that category estimate with the overall training-set average, reducing overfitting when categories are rare. section 6.1.3

Use TargetEncoder when categorical features have many distinct values:

  • Encode high-cardinality fields such as neighborhood or binned geographic coordinates instead of expanding them into thousands of one-hot columns. section 6.1.3
  • Set a smoothing value above zero; choose it experimentally, or consider the James-Stein encoder when stronger variance assumptions are acceptable. section 6.1.3

Geographic categories

Binning latitude and longitude into a 100 × 100 grid and combining the bins creates a coordinates feature with 2,259 distinct values; TargetEncoder can convert this high-cardinality feature into target-informed values.

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Explains posterior-prior blending, smoothing selection, and pipeline configuration, but does not derive the encoding mathematically.
Examples
Uses neighborhood values and grid-binned latitude-longitude coordinates, including a 2,259-level coordinates feature.
What it assumes
Assumes categorical features and regression or classification targets, while defining posterior, prior, and smoothing through practical examples.
What you can do after
Apply it via category_encoders in a mixed categorical pipeline and tune smoothing through experimentation.
How it connects
Connects target encoding with one-hot encoding, the James-Stein encoder, categorical preprocessing, and XGBoost.
Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
liveBookmanning.com
Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.