Log-likelihood

As explained in Data Without Labels

A log-likelihood function is the logarithm of the likelihood of observed data as a function of model parameters.

A log-likelihood function is the logarithm of the likelihood of observed data, evaluated as model parameters change. It provides an objective for fitting probabilistic models. section 5.6.1 section 8.9.1

In Gaussian mixture modeling, expectation-maximization repeatedly updates each Gaussian’s covariance, mean, and density using the probability that each point came from that Gaussian. The process continues until convergence, stopping when the log-likelihood is maximized; the proofs are beyond scope, while implementation is more straightforward. section 5.6.1

Log-likelihood also guides generative neural-network training:

• Optimizes the probability of observed data under a deep belief network’s generative model. section 8.9.1
• Uses contrastive divergence to approximate the gradient of log-likelihood for each restricted Boltzmann machine layer. section 8.9.1
• Supports layer-wise pretraining, allowing successive layers to learn increasingly abstract features without labeled data. section 8.9.1

Gaussian mixture fitting

Assign each data point a probability of belonging to Gaussian j, use those probabilities to update the Gaussian parameters, and repeat until the log-likelihood reaches its maximum. section 5.6.1

(5.10)

Inspect the supplied equations for the Gaussian mixture likelihood and its parameter updates, including covariance, mean, and density. section 5.6.1

How Data Without Labels covers it

How deep it goes

Explains iterative parameter updates and optimization goals, but defers mathematical proofs and detailed implementation.

Examples

Uses Gaussian-component assignments, iterative parameter updates, RBM layers, and Gibbs sampling as concrete examples.

What you can do after

Understand how probabilistic models define training objectives and how algorithms approximate or maximize them.

How it connects

Connects Gaussian mixture models, expectation-maximization, covariance, restricted Boltzmann machines, contrastive divergence, and Gibbs sampling.

liveBookmanning.com

Read Data Without Labels on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Effective Data AnalysisLog-likelihood is the logarithm of the likelihood of observed data under a statistical model and can support comparisons of model fit.
• 

  liveBookmanning.com
  Causal AILog-likelihood is the natural logarithm of the likelihood of observed data under candidate parameter values.

Compare how 3 books explain log-likelihood →

Related concepts

• likelihooduses · A likelihood is the probability of observed data under a specified probabilistic model, regarded as a function of candidate hypotheses or parameter values.
• optimizationused by · Optimization is the systematic selection or adjustment of values, actions, or system designs to improve a specified objective, often subject to constraints.
• DPO lossused by · The DPO loss is a preference-training objective that optimizes a policy to assign higher relative likelihood to preferred responses than to rejected responses compared with a reference policy.
• score functionproduces · A score function is a function of data and parameters whose value or equation quantifies sensitivity to parameter changes and can characterize or estimate those parameters.

This explainer was generated by Manning's knowledge graph from the text of Data Without Labels. The section links point to the parts of the book it draws on.