8 Mixture models: Modeling implicit group memberships
This chapter covers
- Modeling implicit groups with mixture models
- Gaussian mixtures as universal approximators
- Dirichlet processes and the stick-breaking structure to model unknown numbers of groups
In hierarchical models, we assume that our data were grouped in ways we could clearly observe—students in schools, patients in hospitals. But sometimes, the grouping structure isn’t visible to us. Each datapoint can still belong to some group, but we don’t know which one.
For example, we’re analyzing customer purchase behavior on an online shopping platform. Even though these groups are not explicitly defined by any label, we can expect shoppers to belong to different groups such as bargain hunters and brand loyalists. Here, a single probability distribution, however flexible, wouldn’t be able to capture this mixed trend.
This is where mixture models come in. Instead of assigning observations to known categories, mixture models infer the hidden subpopulations directly from the data. They let us make statements like, “it looks like there are different types of customers” or “there seem to be two patterns of eruptions,” even if we didn’t label those groups ahead of time.