liveBook · Manning

Markov chain monte carlo

Also called MCMC

Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution.

How 4 Manning books explain it

Grokking Bayes

Markov chain Monte Carlo is a family of sampling algorithms that constructs a Markov chain with a target probability distribution as its stationary distribution.

How it is used
Approximates Bayesian posteriors through samples when a usable closed-form posterior is unavailable.
How deep it goes
Works through Metropolis–Hastings steps and a NumPy implementation, then gives sampling checks without proving the algorithm’s guarantee.
Examples
Uses a tea-versus-coffee preference model, a Beta(4, 3) sampling comparison, and Bayesian neural-network prediction intervals.
What you can do after
Build it from scratch with NumPy, apply it via PyMC, and diagnose samples using ArviZ convergence and efficiency statistics.

Read how Grokking Bayes explains markov chain monte carlo →

Machine Learning Algorithms in Depth

Markov chain Monte Carlo is a Monte Carlo methodology that samples a Markov chain to approximate a target probability distribution, typically a posterior distribution.

How it is used
Approximates high-dimensional Bayesian posterior distributions through samples from a Markov chain.
How deep it goes
Defines the stationary-distribution principle and sketches Gibbs and Metropolis–Hastings updates without specifying a complete sampler implementation.
Examples
Uses inverse-CDF exponential sampling, π estimation, a binomial tree model, and a self-avoiding random walk as sampling examples.
What you can do after
Evaluate/understand it as a way to estimate posterior quantities and distinguish sampling from variational approximation.

Read how Machine Learning Algorithms in Depth explains markov chain monte carlo →

Causal AI

Markov chain Monte Carlo is a family of sampling algorithms that generate a Markov chain whose limiting distribution is a target probability distribution.

How it is used
Supports probabilistic inference after causal-model transformations, including interventions, conditioning, and counterfactual queries.
How deep it goes
Explains accept/reject chains and limiting convergence, then applies MCMC within a causal inference workflow.
Examples
Uses an online gaming query involving intervention E and outcome IE to demonstrate sampling from causal distributions.
What you can do after
Apply it via Pyro or PyMC to sample from distributions defined by transformed causal models.

Read how Causal AI explains markov chain monte carlo →

Timeless Algorithms, The Foundational Ideas

Markov chain Monte Carlo is a family of algorithms that uses Markov chains to generate samples from probability distributions.

How it is used
Approximates Bayesian inference when complex posterior distributions cannot be computed analytically.
How deep it goes
Explains repeated sampling and its role in posterior estimation, but does not detail specific algorithms or sampling mechanics.
Examples
Names genomics, climate science, financial modeling, and engineering design as application areas.
What you can do after
Understand how repeated sampling can estimate a posterior when exact solutions are impractical.

Read how Timeless Algorithms, The Foundational Ideas explains markov chain monte carlo →

What to know first

Related concepts

Questions about markov chain monte carlo

What is markov chain monte carlo?

Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution.

Which Manning books explain markov chain monte carlo?

Grokking Bayes, Machine Learning Algorithms in Depth, Causal AI and Timeless Algorithms, The Foundational Ideas.

What should I know before learning markov chain monte carlo?

Manning books that teach markov chain monte carlo usually introduce posterior probability, conditional probability, gibbs sampling and Markov chain first.

What is the difference between markov chain monte carlo and variational inference?

Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution. Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.

What kinds of markov chain monte carlo are there?

Kinds of markov chain monte carlo include metropolis-hastings sampling.

What is markov chain monte carlo also called?

Markov chain monte carlo is also called MCMC.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.