Posterior probability

As explained in Machine Learning Algorithms in Depth

A posterior distribution is the probability distribution of unknown quantities after incorporating observed data into prior beliefs.

A posterior distribution assigns probabilities to unknown model parameters after observing data. Let θ denote the parameters and x the observations. Bayes’ rule gives p(θ|x) = p(x|θ)p(θ)/p(x): the prior p(θ) represents initial beliefs, while the likelihood p(x|θ) measures how the model accounts for the data. As new observations arrive, a posterior can become the prior for the next update. section 1.4

The denominator p(x) = ∫p(x|θ)p(θ)dθ normalizes the distribution. This integral can be intractable in high-dimensional parameter spaces. The choice of prior matters: both its parameter values and its functional form affect the result. A conjugate prior has the same form as the posterior and permits a closed-form update. section 1.4

When a posterior is difficult to compute directly, two approaches are:

• Markov chain Monte Carlo (MCMC): sample from parameter space using a chain whose stationary distribution is the target posterior, then use the samples to estimate quantities of interest. Gibbs sampling and Metropolis–Hastings are two methods. section 1.4.1 section 2.9
• Variational inference (VI): choose a tractable distribution q and optimize it to approximate the posterior. section 1.4.1

Dirichlet–multinomial update

Compute the posterior using a Dirichlet prior and a multinomial likelihood, then examine how its shape changes with the posterior counts. This exercises a conjugate-prior update rather than a sampling approximation. section 8.4 section 1.4

Figure 1.4 A Bayes engine showing the transformation of a prior to a posterior as more data is observed

The Bayes engine depicts repeated updates: each new observation transforms a prior into a posterior that can inform the next update. section 1.4

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives the prior–likelihood relationship and normalization, then motivates approximation when high-dimensional integration is intractable.

Examples

Uses a sequential Bayes-engine figure and a Dirichlet–multinomial posterior exercise.

What it assumes

Assumes familiarity with probability distributions and integration while defining the prior, likelihood, and posterior.

What you can do after

Practice/perform a Dirichlet–multinomial posterior calculation and distinguish MCMC sampling from variational approximation.

How it connects

Connects Bayesian inference to conjugate priors, probabilistic graphical models, Markov chain Monte Carlo, and variational inference.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA posterior probability is the probability assigned to an event or hypothesis after incorporating observed evidence.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningThe updated distribution over model parameters after combining prior and likelihood.
• 

  liveBookmanning.com
  Fabulous Adventures in Data Structures and AlgorithmsA posterior distribution is a probability distribution representing beliefs about an unknown quantity after incorporating observed evidence.
• 

  liveBookmanning.com
  Bayesian Optimization in ActionA posterior distribution is a probability distribution representing updated beliefs about unknown quantities after incorporating observed data.
• 

  liveBookmanning.com
  Build a Robo-Advisor with Python (From Scratch)A posterior distribution is a probability distribution representing updated beliefs about an unknown quantity after incorporating data or observations.
• 

  liveBookmanning.com
  AI-Powered SearchA posterior distribution is a probability distribution representing an updated belief after incorporating observations into a prior distribution.
• 

  liveBookmanning.com
  Causal AIA posterior distribution is the conditional probability distribution of unknown quantities given observed data and prior assumptions.

Compare how 8 books explain posterior probability →

Related concepts

• bayesian statisticsproduced by · Bayesian statistics is a framework for statistical inference that combines prior probability distributions with observed-data likelihoods to update probabilities over hypotheses or unknown quantities.
• prior probabilityproduced by · A prior probability distribution represents beliefs or assumptions about an unknown quantity before relevant observations or evidence are incorporated.
• Bayes' theoremproduced by · Bayes' theorem is a probability rule that expresses a conditional probability in terms of a prior probability, a likelihood, and the evidence.
• markov chain monte carloused by · Markov chain Monte Carlo is a family of sampling algorithms that generate samples using a Markov chain whose stationary or limiting distribution is a target probability distribution.
• fully bayes parameter estimationproduced by · Fully Bayesian parameter estimation combines a prior distribution with observed-data likelihood to produce a posterior distribution over unknown parameters.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.