A geometric transformation maps data from one geometric space to another. In deep learning, vectorized inputs pass through layers that apply simple tensor-based transformations; together, those layers form a complex transformation from input space to target space. section 20.1.3 section 2.6
Uncrumpling two classes
Imagine red and blue sheets of paper crumpled together into a ball. Each layer makes a small geometric adjustment, gradually uncrumpling the ball until the two classes become cleanly separable. The network learns the adjustments through its weights. section 2.3.6 section 20.1.3
Key properties shape what neural networks can represent:
- Layers apply elementary transformations whose composition can represent a highly complicated mapping. section 2.3.6
- Weights parameterize the transformation and are updated according to how well the model performs. section 20.1.3 section 2.6
- Differentiability makes gradient-based learning possible, requiring a smooth and continuous morphing from inputs to outputs. section 20.1.3 section 2.6
- Continuous transformations suit perceptual pattern recognition but poorly express discrete, step-by-step logic such as carrying in addition. section 19.1.3
Figure 2.13 Uncrumpling a complicated manifold of data
Visualizes a complicated data manifold being transformed until its classes become separable. section 2.3.6
