Gaussian mixture model

As explained in Data Without Labels

A probabilistic model that represents data as a weighted combination of Gaussian probability distributions.

A Gaussian mixture model (GMM) is a soft clustering method for data whose groups may overlap. Instead of locating clusters by centroids alone, it fits several Gaussian distributions and assigns each data point a probability of belonging to each one. section 5.6.1 section 5.9

A Gaussian distribution has a mean that locates its center and a variance that affects its shape. For multidimensional data, the mean becomes a vector and the covariance matrix describes variation and relationships among inputs. A GMM fits a mean, covariance, and weight for each distribution; unlike k-means, it uses more than the mean. section 5.6 section 5.6.1

Figure 5.20 A Gaussian distribution is one of the most famous distributions. Observe how the values of mean and standard deviation are changed and their effect on the corresponding curve.

Compare the curves to see how changing a Gaussian distribution’s mean and standard deviation changes its position and shape. section 5.6

Expectation–maximization (EM) estimates the mixture parameters iteratively:

• Start with current estimates of the distributions and their parameters. section 5.6.1
• Estimate each point’s probability of belonging to each distribution. section 5.6.1
• Update the model parameters using those estimates, then repeat until the values fit the data well enough. section 5.6.1

An overlapping point

Suppose a point could have been generated by either of two Gaussian distributions. A single definite label hides that uncertainty; GMM estimates its probability of belonging to each cluster. This makes it useful when groups overlap. section 5.6.1 section 5.9

How Data Without Labels covers it

How deep it goes

Explains Gaussian parameters and the iterative EM procedure, then points toward Python implementation without specifying a library here.

Examples

Uses changing Gaussian curves and a point that could come from two distributions to illustrate shape and uncertain membership.

What it assumes

Builds from an introduction to the normal distribution and assumes no advanced mathematics, while drawing on basic clustering concepts.

What you can do after

Evaluate/understand GMM as a clustering option for overlapping data and interpret its probabilistic assignments.

How it connects

Connects Gaussian distributions, covariance, expectation–maximization, soft clustering, and k-means.

liveBookmanning.com

Read Data Without Labels on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA probabilistic model representing a distribution as a weighted combination of Gaussian components.
• 

  liveBookmanning.com
  Outlier Detection in PythonA Gaussian mixture model represents a probability distribution as a weighted combination of multiple Gaussian component distributions.

Compare how 3 books explain Gaussian mixture model →

Related concepts

• soft clusteringuses · Soft clustering is a clustering technique that assigns each item graded or probabilistic membership across multiple clusters rather than exclusive membership in one.
• generative modelingkind of · Generative modeling is a machine-learning approach that learns patterns or a data distribution to generate new samples resembling the data.
• normal distributionuses · A normal distribution is a continuous probability distribution with a symmetric, bell-shaped density determined by its mean and variance.
• expectation maximizationused by · Expectation maximization is an iterative algorithm for estimating model parameters with latent or unobserved variables by alternating latent-variable estimation with parameter optimization.
• K-means clusteringcontrasts with · K-means clustering partitions data into a specified number of groups by assigning observations to representative centers and iteratively updating those centers to reduce within-group distances.

This explainer was generated by Manning's knowledge graph from the text of Data Without Labels. The section links point to the parts of the book it draws on.