A mixture weight πk is a nonnegative coefficient that assigns the kth Gaussian component its prior selection probability in a Gaussian mixture model (GMM). The weights satisfy ∑k πk = 1, so they form a categorical distribution over components. In the GMM density, πk scales the kth Gaussian contribution: p(x) = ∑k πk 𝒩(x; μk, Σk). section 6.9.1 section 6.9.2
Interpret mixture weights in three complementary ways:
- As population proportions, πk can represent the fraction of training data associated with subclass k. section 6.9.2
- As a sampling rule, select component k with probability πk, then draw a point from its Gaussian distribution. section 6.9.2
- As a geometric control, weights regulate the relative heights of PDF hills and influence the density of samples around component clusters; means set locations and covariances set shapes. section 6.9.1
Changing the balance between two components
Keep two Gaussian components fixed and change only their weights. With π1 = 0.5 and π2 = 0.5, both components contribute equally. With π1 = 0.7 and π2 = 0.3, the first component produces a relatively higher density hill and more samples; reversing the weights reverses that emphasis. For three components, π1 = 0.4, π2 = 0.4, and π3 = 0.2 assigns the third component half the prior mass of either of the first two. section 6.9.1
During maximum-likelihood fitting on unlabeled data, mixture weights are estimated together with component means and covariances. Because the likelihood equations have no closed-form solution for a GMM, an iterative E-step/M-step procedure updates them until the likelihood stops increasing; the M-step sets πk to the normalized effective membership count Nk/n. section 6.9.4
Listing 6.8 Gaussian mixture model distribution
from torch.distributions.mixture_same_family import MixtureSameFamily #1
pi = Categorical(torch.tensor([0.4, 0.4, 0.2])) #2
mu = torch.tensor([[175.0, 70.0], [152.0, 55.0], [135.0, 40.0]]) #3
sigma = torch.tensor([[[30.0, 20.0], [20.0, 30.0]], #4
[[50.0, 0.0], [0.0, 10.0]],
[[20.0, 0.0], [0.0, 20.0]]])
gaussian_components = MultivariateNormal(mu, sigma) #5
gmm = MixtureSameFamily(pi, gaussian_components) #6
A PyTorch construction makes the mixture weight concrete: Categorical([0.4, 0.4, 0.2]) supplies component-selection probabilities for MixtureSameFamily. section 6.9.2
