liveBook · Manning

Mixture model

As explained in Grokking Bayes

A mixture approximating family uses combinations of distributions to represent richer variational approximations.

A mixture model represents data as coming from several hidden groups. Instead of assigning observations to known categories, it infers subpopulations whose distributions combine to explain the observed pattern. section 8.0 section 8.12

Think of shoppers as bargain hunters or brand loyalists, or eruption waits as short or long. Each component has a weight, center, and width; together, these describe the overall distribution. Bayesian treatment represents these quantities as uncertain and can estimate each observation’s membership probabilities. section 8.0 section 8.4 section 8.8

Old Faithful waiting times

Standardized waiting times show two modes: short waits and long waits. A two-component Gaussian mixture learns their centers, spreads, and prevalence, then estimates how likely a particular waiting time belongs to either cluster. Posterior predictive samples provide a check that the fitted mixture can reproduce the observed distribution. section 8.3 section 8.8

with pm.Model(
    coords={"cluster": np.arange(K)}  #A
) as dp_model:
    beta = pm.Beta(
        "beta", alpha=1, beta=1, dims="cluster"
    )                                 #B
    w = pm.Deterministic(
        "w", stick_breaking(beta), dims="cluster"
    )                                  #C

Inspect the PyMC setup for a truncated stick-breaking mixture, where Beta variables generate component weights. section 8.10

When the number of groups is unknown, stick-breaking supplies a prior that allows infinitely many components while leaving only a few with meaningful weight in practice. Computation truncates the process at a finite upper bound; the concentration parameter controls whether weight favors fewer large clusters or many smaller ones. section 8.10 section 8.12

How Grokking Bayes covers it

How deep it goes
Explains components, Bayesian uncertainty, membership probabilities, posterior checks, and truncated stick-breaking, stopping short of deeper derivations.
Examples
Uses online shoppers, Old Faithful eruption waits, gene expression, customer segmentation, and household consumption classes.
What it assumes
Assumes basic Bayesian model concepts and introduces mixture components through standardized data and concrete distributions.
What you can do after
Apply it via PyMC and Python to fit components, inspect weights, calculate membership probabilities, and assess posterior predictive fit.
How it connects
Connects Bayesian models, Gaussian mixtures, cluster membership, posterior predictive distributions, hierarchical models, and stick-breaking processes.

Also explained in

Compare how 2 books explain mixture model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.