Categorical data contains values treated as discrete labels rather than ordered magnitudes. Unlike continuous values, differences between categories have no precise mathematical meaning; unlike ordinal values, categories do not imply a meaningful ranking. section 4.3.2
Common representations include:
- One-hot encoding represents each class with a vector, such as [1, 0] for airplane and [0, 1] for bird. The same pattern extends to larger class sets such as CIFAR-10. section 7.2.3
- Embeddings represent categories with learned numeric vectors, avoiding the cumbersome representation of large one-hot vectors. section 4.5.5
Classifying birds and airplanes
A classifier should treat airplane and bird as categories, not as numeric class magnitudes. It can output two probabilities corresponding to one-hot targets such as [1, 0] and [0, 1]. For binary classification, a single probability with sigmoid and binary cross-entropy is also possible. section 7.2.3
