Categorical encoding transforms category-valued features into representations that machine-learning models can process. One-hot encoding is one option; ordinal encoding, mean target encoding, and the hashing trick provide alternatives for different data shapes and risks. section 11.1.2
Use the encoding choice to match the modeling pipeline:
- Apply one-hot encoding as basic feature engineering, including before a logistic regression model. section 10.1.1
- Consider ordinal encoding when categories have an inherent order, or the hashing trick for large-scale categorical data. section 11.1.2
- Treat mean target encoding cautiously because using target information can introduce data leakage. section 11.1.2
From feature engineering to a model
A simple logistic-regression system can combine feature scaling, one-hot encoding, and model inference as an ordered pipeline. In a larger ETA system, categorical and other tabular features may instead become discrete tokens and learnable embeddings for a transformer. section 10.1.1 section 11.1.2
Figure 11.1 An overview of the DeepETA model pipeline: example of combining base feature engineering and a deep learning model (source: https://mng.bz/0MoN)
Inspect the DeepETA pipeline to see how base feature engineering connects categorical and other inputs to a deep learning model. section 11.1.2
