An embedding transforms high-dimensional data into a lower-dimensional representation while preserving selected characteristics or relationships. Unlike ordinary categorical encoding, it is typically learned by a neural-network model. section 6.1.3
For tabular data, embeddings are especially useful when a categorical feature has many distinct values. One-hot encoding can create sparse, memory-intensive matrices, while ordinal encoding may not expose useful partitions to tree models. Embeddings are therefore associated most strongly with deep-learning approaches. section 2.2.5 section 6.1.3
Categorical embeddings in a tabular model
A deep-learning model can represent a high-cardinality categorical feature with a learned embedding rather than expanding every category into a separate binary column. This approach belongs alongside shallow neural networks and tabular frameworks such as fastai Tabular, PyTorch Tabular, TabNet, SAINT, and DeepTables. section 1.3 section 6.1.3
