liveBook · Manning

Marginal likelihood

Also called evidence

Marginal likelihood, or Bayesian evidence, is the probability or density of observed data under a model, averaged over its unknown parameters or latent variables according to their prior distributions.

How 3 Manning books explain it

Retrieval Augmented Generation, The Foundational Ideas

Marginal likelihood is the probability of an observed outcome after summing or averaging over possible latent variables or retrieved documents.

How it is used
Links document retrieval to masked-word prediction by treating the retrieved document as a latent variable in REALM’s training objective.
How deep it goes
Works through probability terms, log-sum-exp code, and a numerical loss example; stops short of runnable end-to-end REALM pretraining.
Examples
Uses a three-document NumPy calculation and a rare-fact masked-word prediction to show how retrieved evidence changes probabilities.
What you can do after
Evaluate/understand how retrieval probabilities affect prediction loss and why REALM trains its retriever for task utility.

Read how Retrieval Augmented Generation, The Foundational Ideas explains marginal likelihood →

Grokking Bayes

The probability of observed data under a model after integrating or summing over the model's uncertain parameters according to their prior distribution.

How it is used
Compares how likely observed data are under competing models by averaging over each model’s hypotheses.
How deep it goes
Defines prior-weighted averaging, distinguishes discrete sums from continuous integration, and works through a beta-binomial implementation.
Examples
Uses tea-preference counts and a 500-trial conversion-rate comparison, alongside a SciPy calculation.
What you can do after
Apply it via SciPy’s comb and beta functions to compare beta-binomial models and calculate Bayes factors.

Read how Grokking Bayes explains marginal likelihood →

Bayesian Optimization in Action

A marginal likelihood is the probability or density of observed data after integrating over latent variables or model parameters.

How it is used
Uses data fit to select Gaussian-process mean and covariance hyperparameters.
How deep it goes
Defines likelihood-based fitting, demonstrates parameter comparison, and gives an optimization loop without deriving the probability calculation.
Examples
Compares constant means of 0 and 2 and tunes length and output scales on a three-point training dataset.
What you can do after
Apply it via GPyTorch and Adam to optimize Gaussian-process mean or covariance parameters.

Read how Bayesian Optimization in Action explains marginal likelihood →

Also covered in

What to know first

Related concepts

Questions about marginal likelihood

What is marginal likelihood?

Marginal likelihood, or Bayesian evidence, is the probability or density of observed data under a model, averaged over its unknown parameters or latent variables according to their prior distributions.

Which Manning books explain marginal likelihood?

Retrieval Augmented Generation, The Foundational Ideas, Grokking Bayes and Bayesian Optimization in Action.

What should I know before learning marginal likelihood?

Manning books that teach marginal likelihood usually introduce closed form, hypothesis and gradient descent first.

What is marginal likelihood also called?

Marginal likelihood is also called evidence.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.