Unsupervised learning uses observations without training labels to discover patterns or learn representations. Unlike supervised learning, it has no target output y for each input x. The question is not whether a prediction matches a supplied answer, but what structure the observations reveal. section 8.0 section 1.1
Different algorithms seek different kinds of structure:
- Clustering groups observations. Dirichlet process K-means extends K-means so the number of clusters can grow with the data. section 8.5
- Density modeling represents a distribution with a Gaussian mixture model. Expectation-Maximization alternates between inferring missing values given current parameters and updating the means, covariances, and mixture proportions. section 8.5
- Dimensionality reduction projects high-dimensional observations into fewer dimensions while retaining useful characteristics. PCA maximizes projected variance; t-SNE can produce results sensitive to perplexity and successive runs. section 8.5
Unsupervised learning also includes neural representations. An autoencoder learns to reconstruct its input; its bottleneck can serve for dimensionality reduction or feature extraction. A variational autoencoder (VAE) maps observations to a latent representation and decodes that representation into a reconstruction. section 11.7
Detecting an unusual time-series signal
Train an LSTM-VAE on nonanomalous sensor signals without anomaly labels. For each signal xᵢ, the model estimates Gaussian parameters μᵢ and σᵢ and evaluates log p(xᵢ | μᵢ, σᵢ). Training maximizes average log likelihood while shaping the joint latent space toward a standard normal distribution through KL divergence. A drop in log likelihood can then be thresholded to signal an anomaly. section 11.1.1
Figure 1.2 Unsupervised learning: clusters of data points projected onto 2-dimensional space
A two-dimensional projection makes the clustering goal visible as groups of data points. section 1.1
