A mixture weight is a nonnegative proportion assigned to one component distribution. For K components, the weights πₖ satisfy 0 ≤ πₖ ≤ 1 and sum to one, so they define a convex combination of component densities. section 2.1 section 8.2
In a Gaussian mixture model, weights accompany component means and covariances. They determine how strongly each Gaussian contributes to the overall density; fitting the model therefore estimates πₖ alongside μₖ and the covariance parameters. section 2.1 section 8.2
A valid set of mixture weights must:
- Remain nonnegative, or strictly positive when the formulation requires πₖ > 0. section 2.1
- Sum to one, creating the normalization constraint and reducing the number of free parameters by one. section 2.1
- Use a softmax output when predicted by a neural network, ensuring that the network outputs form normalized mixture weights. section 11.2.1
Iris Gaussian mixture
With three Iris species, a Gaussian mixture uses K = 3 component distributions and three mixture weights. The weights, together with the component means and covariances, describe the density structure visible in the data. section 2.1
Mixture density network
A mixture density network predicts mixture weights from input data. An MLP uses softmax activations for the weights, learns means and variances jointly, and trains by maximizing log likelihood or minimizing negative log likelihood. section 11.2.1
Figure 11.7 A multioutput neural network used to learn Gaussian parameters
Shows the multi-output neural-network structure used to predict Gaussian parameters, including mixture weights. section 11.2.1
