Generation learns or fits a probability distribution to training data, then samples from that distribution to create new data resembling the examples. A model can also evaluate the probability of a new instance. The central idea is to find a density whose sample cloud overlaps the data of interest. section 5.7 section 13.0
Generation differs from discriminative prediction in what it models:
- A discriminative model learns a decision boundary or emits a class; a generative model assigns probabilities across the input space. section 14.2
- Smooth probability densities can avoid following irregular training-data bends too closely, especially when training data is limited. section 14.2
- A generative classifier can become a discriminative classifier by thresholding the probability it assigns to an input. section 14.2
- Distribution parameters such as expected value, variance, and covariance describe the fitted family and provide geometric intuition for its samples. section 5.7
Generating and evaluating cat images
Given training examples of cat images, fit a density p(x) whose sample cloud overlaps those examples. Sampling p(x) produces additional computer-generated cat images; evaluating p(x) for a new image estimates how likely it is to be a cat image. section 13.0
Figure 14.2 Solid circles indicate training data points (all belonging to the class of interest). The dashed curve indicates the decision boundary separating the class of interest from the class of non-interest. In a generative model, there is no decision boundary. Every point in the space is associated with a probability of belonging to the class of interest (indicated as a heat map in figure 14.2c)
|
(a) A good discriminative classifier—smooth decision boundary |
|
(b) A bad discriminative classifier—irregular decision boundary |
|
(c) Generative model—no decision boundary (heat map indicates the probability density) |
Compare a smooth generative probability landscape with good and overfit discriminative decision boundaries. section 14.2
