A high-resolution image may contain millions of pixels, many of them nearly identical to their neighbors. Representation learning transforms such raw data into a structured form that retains useful information while reducing redundancy and noise. The resulting representation can make data easier to process, search, visualize, or use in downstream tasks. section 5.4.2 section 5.4
Neural networks commonly learn representations during training, although representation learning is broader than any one model. Earlier methods often relied on features chosen by experts; deep networks instead learn successive transformations of the input. In image recognition, these transformations may move from pixels to edges, shapes, and eventually objects. section 5.4.2 section 11.1
The information-theoretic perspective connects representation learning to source coding: both seek a compact description that preserves informative content while discarding predictable variation. But compactness alone is not the goal. What can safely be discarded depends on what the representation must support; information useful for reconstruction, for example, need not be identical to information useful for classification. section 5.4.2 section 11.5.2
Figure 5.3 Representation learning as information-preserving compression. An encoder transforms a high-dimensional image into a compact latent representation by preserving the image's informative structure while reducing redundant detail. Although many original pixels are no longer represented explicitly, the latent representation retains the information needed for a decoder to reconstruct an image that closely resembles the original.

The encoder compresses an image into a latent representation, from which a decoder reconstructs an image resembling the original. section 5.4.2
A learned representation can also change the geometry of a prediction problem. Hidden layers may transform observations that require a curved boundary in the input space into coordinates where a straight boundary separates the classes. The nonlinearity lies in the transformation, not necessarily in the final boundary. section 11.5.2
Figure 11.3 A nonlinear transformation reorganizes the same six observations so that classes requiring a curved boundary in the input space can be separated by a straight boundary in the learned representation.

The same circles and squares become linearly separable after their coordinates are transformed. section 11.5.2
Easier separation of training observations is not enough on its own. The transformation improves generalization only when it captures relationships that also occur in unseen observations. section 11.5.2




