A high-cardinality categorical feature has many distinct, nonnumeric categories. ZIP code illustrates the challenge: the United States contains 41,692 unique ZIP codes, far more than its 50 states. section 8.0
High-cardinality string features often contain duplicate meanings, alternate spellings, and typos. Exact matching creates still more categories, while one-hot encoding expands the feature space and ordinal encoding may invent an unjustified order. section 8.4
A practical workflow combines diagnosis, encoding choice, and held-out evaluation:
- Count unique values, then separate low-cardinality categorical columns, high-cardinality columns, and continuous features. section 8.4
- Prefer compact encodings when one-hot columns would become numerous and sparse; use similarity rather than exact matching for related strings. section 8.4 section 8.5
- Compare SimilarityEncoder, MinHashEncoder, and GapEncoder inside a modeling pipeline rather than assuming one method will perform best. section 8.4
- Use a train/test split and compare predictive scores to assess whether the representation preserves useful information. section 8.4
Employee salary data
An employee-salary data set contains 9,211 rows. The employee_position_title feature has 385 categories, compared with 2 for gender and 37 for department_name. Treat the title as high-cardinality, encode it with SimilarityEncoder, MinHashEncoder, or GapEncoder, and compare the resulting XGBoost regressors. Reported R² scores are approximately 0.900 for SimilarityEncoder and MinHashEncoder, and 0.890 for GapEncoder. section 8.4
import pandas as pd
df = pd.read_csv('./data/ch08/employee_salaries.csv')
df.head()
Inspect the employee-salary data directly before counting categories and assigning high-cardinality encoders. section 8.4
