BIC evaluates competing statistical models by balancing goodness of fit against model complexity. With Gaussian mixture models, it helps choose how many components to use rather than selecting a fixed number arbitrarily. section 5.6
A practical workflow fits candidate mixture models with different component counts and retains the candidate associated with the best BIC score. The selected structure can still contain a very small component, so model selection does not by itself establish that every component is substantively meaningful. section 5.6 section 7.8.1
BIC supports several implementation choices:
- Compare candidate Gaussian mixture models across a range of component or cluster counts. section 5.6
- Use the selected model to obtain likelihood-based outlier scores or to support cluster-based scoring. section 5.6 section 7.8.1
- Treat BIC as one model-selection step, alongside checks of component weights, cluster behavior, and stochastic fitting stability. section 5.6
Selecting components for synthetic clusters
A loop fits Gaussian mixture models with one through seven components and records the best BIC. On synthetic data generated around three clusters with additional dispersed points, the selected model contains four components; one has weight 0.03 and contributes low likelihoods even near its center. section 5.6
Listing 5.9 Example of GMMs for outlier detection
from sklearn.datasets import make_blobs
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.mixture import GaussianMixture
from scipy import stats
np.random.seed(0)
vals, _ = make_blobs(n_samples=500, centers=3, n_features=2,
cluster_std=1, random_state=42) #1
df_inliers = pd.DataFrame(vals, columns=["A", "B"])
vals, _ = make_blobs(n_samples=15, centers=2, n_features=2,
cluster_std=10, random_state=42)
df_outliers = pd.DataFrame(vals, columns=["A", "B"])
df = pd.concat([df_inliers, df_outliers])
X = df[['A', 'B']]
best_n_components = -1 #2
best_bic = np.inf
for n_components in range(1, 8):
gmm = GaussianMixture(n_components=n_components, n_init=5,
random_state=42)
gmm.fit(X)
bic = gmm.bic(X)
if bic < best_bic:
best_bic = bic
best_n_components = n_components
gmm = GaussianMixture(n_components=best_n_components,
n_init=5, random_state=42) #3
gmm.fit(X)
score = gmm.score_samples(X)
df['Score'] = score
pct_threshold = np.percentile(score, 3.0) #4
df['Outlier'] = df['Score'] < pct_threshold
sns.scatterplot(data=df[df['Outlier'] == False], x='A', y='B', alpha=0.3)
sns.scatterplot(data=df[df['Outlier'] == True], x='A', y='B',
s=150, marker='*')
plt.show()
Inspect the implementation pattern for fitting candidate Gaussian mixture models, comparing BIC values, and retaining the selected component count. section 5.6
