LeaveOneOutEncoder is a supervised categorical encoder that replaces a category with a target statistic while excluding the row currently being encoded. For a training row, its category value is computed from the other matching rows rather than from its own label. This preserves more modeling data than reserving a separate encoding hold-out set. section 8.1.3
Figure 8.6 LOO target encoding is applied to the training data to avoid creating a wasteful hold-out encoding set. Instead of holding out a subset of the data, only the example being encoded is held out. Test data is encoded using greedy target encoding as before.
The diagram contrasts holding out one row at a time for training encodings with greedy target encoding for test data. section 8.1.3
Use it with target-encoded categorical columns as follows:
- Split features and target before preprocessing, so the encoder is fitted only on training data. section 8.1.4
- Place `category_encoders.LeaveOneOutEncoder` in a categorical preprocessing pipeline and set `cols` to the categorical feature names. section 8.1.4
- Set `sigma` to add noise intended to reduce overfitting; recommended values are 0.05 to 0.6. section 8.1.4
- Transform the held-out test features through the fitted preprocessing pipeline, then train and evaluate the downstream model. section 8.1.4
Credit-approval preprocessing
A pipeline encodes four categorical fields with `ce.LeaveOneOutEncoder(cols=cat_features, sigma=0.4)` and retains 14 features, unlike one-hot encoding's 38. A RandomForestClassifier then reaches 90.580% hold-out accuracy in the illustrated run. section 8.1.4
Leave-one-out encoding mitigates leakage and prediction shift without discarding a dedicated encoding subset, but it does not completely remove prediction shift. Ordered target statistics provide a permutation-based alternative for addressing that remaining issue. section 8.2.1 section 8.5
