Sparsity describes a representation dominated by zeros or empty values. In tabular machine learning, one-hot encoding can create many binary columns; when most entries are zero, learning may become more difficult, especially for neural networks and online algorithms. section 4.2.1
The practical trigger is often categorical cardinality. Encoding a feature with more than 20 unique values can produce excessive columns and a sparse dataset, so separate low-cardinality features from high-cardinality ones before encoding. section 4.2.1
Categorical features in a listings dataset
A listings dataset contains five neighbourhood groups, 221 neighbourhoods, and three room types. Treat neighbourhood_group and room_type as low-cardinality categorical features, while neighbourhood is high-cardinality and may require another encoding strategy rather than direct one-hot encoding. section 4.2.1
Use sparsity as a modeling and preprocessing signal:
- Measure sparsity through the percentage of zero values among numeric entries. section 4.3.6
- Expect increasing memory and computational pressure as the number of binary features grows. section 4.3.6
- Consider online learning or deep learning earlier when datasets are very sparse, particularly at large scale. section 4.3.6
- Use target encoding for categorical features with too many unique values when direct one-hot encoding becomes excessive. section 4.2.1
