Representation learning automates the discovery of useful representations for feature detection, reducing reliance on manually engineered features. It is particularly relevant when models learn from images, text, audio, or video rather than only prepared tabular variables. section 9.2
Representation learning appears in both supervised and unsupervised settings:
- Supervised feature learning commonly uses neural networks to learn representations useful for prediction. section 9.2
- Unsupervised feature learning includes matrix factorization, clustering algorithms, and autoencoders. section 9.2
Autoencoder-based feature learning
An autoencoder learns a compressed representation through an encoder, a bottleneck, and a decoder that reconstructs the input. Training specifies the code size and loss function. The learned representation can support dimensionality reduction, anomaly detection, data compression, or synthetic-data generation, although reconstruction can lose information and the result is dataset-specific. section 9.11
