Label encoding transforms categorical values into numeric codes. Also called ordinal encoding, it maps string values in a feature to ordered numeric values; the assignments can be arbitrary, so the codes should not be treated as a correlation-friendly numeric measurement. section 2.4.3 section 6.1.3
Choose encoding with both model type and category cardinality in mind:
- Use ordinal encoding with complex tree-based models such as random forests and gradient-boosted trees, which can recursively split categories into useful prediction partitions. section 6.1.3
- Prefer one-hot encoding for categorical features used by linear models; ordinal encoding does not always work properly there. section 6.1.3 section 7.2.2
- Treat high-cardinality features cautiously: one-hot encoding can create memory-heavy sparse matrices, while trees may struggle to split many ordinal codes meaningfully. section 6.1.3
- Do not use correlation to compare differently encoded categorical features; use association measures such as Cramer's V when correlation is inapplicable. section 2.4.3
Encoding an Airbnb feature pipeline
For a tree-based XGBoost workflow, ordinally encode neighbourhood_more_than_30 and type_of_accommodation, one-hot encode room_type, and target encode the higher-cardinality coordinates feature. This shared pipeline limits maintenance effort, though separate model-specific pipelines may improve performance when evidence justifies them. section 7.2.2
Listing 7.13 Defining column transformations
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from category_encoders.target_encoder import TargetEncoder
onehot_encoder = OneHotEncoder(handle_unknown='ignore') #1
ordinal_enconder = OrdinalEncoder(handle_unknown="use_encoded_value",
unknown_value=np.nan) #2
target_encoder = TargetEncoder(
cols=target_encoding,
handle_unknown="value",
smoothing=0.5
) #3
column_transform = ColumnTransformer(
[('onehot_encoding', onehot_encoder, onehot_encoding),
('ordinal_encoding', ordinal_enconder, ordinal_encoding),
('target_encoding', target_encoder, target_encoding),
('numeric', 'passthrough', numeric),
('binary', 'passthrough', binary)], #4
remainder='drop', #5
verbose_feature_names_out=True, #6
sparse_threshold=0.0) #7
A ColumnTransformer combines OneHotEncoder, OrdinalEncoder, and TargetEncoder so each feature group receives its chosen transformation before modeling. section 7.2.2
