Hold-out target encoding replaces each category with a target statistic computed from a separate encoding subset. Unlike greedy target encoding, it keeps the labels used for encoding independent from the examples being encoded, preventing target leakage and prediction shift. section 8.1.3
Apply the technique by separating the data into three roles:
- Create a training set, a hold-out encoding set, and a hold-out test set. section 8.1.3
- Compute category target statistics using only the hold-out encoding set. section 8.1.3
- Use those same statistics to encode both the training and test sets, preserving independence and avoiding prediction shift. section 8.1.3
- Discard the encoding set after computing the statistics, accepting that part of the available data cannot train the model. section 8.1.3
Encoding the category red
Suppose color is encoded by the mean cost associated with each category. Greedy encoding uses all training labels, which can leak information into evaluation. Hold-out encoding calculates red's mean cost from the dedicated encoding subset, then applies that value to red examples in both training and testing. section 8.1.3
Figure 8.5 Hold-out encoding partitions the available data into three sets: training and test, as usual, and a third hold-out test set to be used exclusively for encoding with target statistics. This avoids both target leakage and distribution shift.
A three-way partition diagram shows how the encoding subset remains separate from the training and test sets while supplying both with target statistics. section 8.1.3
