Markov chain monte carlo
Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution.
Grokking Bayes
Markov chain Monte Carlo is a family of sampling algorithms that constructs a Markov chain with a target probability distribution as its stationary distribution.
- How it is used
- Approximates Bayesian posteriors through samples when a usable closed-form posterior is unavailable.
- How deep it goes
- Works through Metropolis–Hastings steps and a NumPy implementation, then gives sampling checks without proving the algorithm’s guarantee.
- Examples
- Uses a tea-versus-coffee preference model, a Beta(4, 3) sampling comparison, and Bayesian neural-network prediction intervals.
- What you can do after
- Build it from scratch with NumPy, apply it via PyMC, and diagnose samples using ArviZ convergence and efficiency statistics.
Machine Learning Algorithms in Depth
Markov chain Monte Carlo is a Monte Carlo methodology that samples a Markov chain to approximate a target probability distribution, typically a posterior distribution.
- How it is used
- Approximates high-dimensional Bayesian posterior distributions through samples from a Markov chain.
- How deep it goes
- Defines the stationary-distribution principle and sketches Gibbs and Metropolis–Hastings updates without specifying a complete sampler implementation.
- Examples
- Uses inverse-CDF exponential sampling, π estimation, a binomial tree model, and a self-avoiding random walk as sampling examples.
- What you can do after
- Evaluate/understand it as a way to estimate posterior quantities and distinguish sampling from variational approximation.
Read how Machine Learning Algorithms in Depth explains markov chain monte carlo →
Causal AI
Markov chain Monte Carlo is a family of sampling algorithms that generate a Markov chain whose limiting distribution is a target probability distribution.
- How it is used
- Supports probabilistic inference after causal-model transformations, including interventions, conditioning, and counterfactual queries.
- How deep it goes
- Explains accept/reject chains and limiting convergence, then applies MCMC within a causal inference workflow.
- Examples
- Uses an online gaming query involving intervention E and outcome IE to demonstrate sampling from causal distributions.
- What you can do after
- Apply it via Pyro or PyMC to sample from distributions defined by transformed causal models.
Timeless Algorithms, The Foundational Ideas
Markov chain Monte Carlo is a family of algorithms that uses Markov chains to generate samples from probability distributions.
- How it is used
- Approximates Bayesian inference when complex posterior distributions cannot be computed analytically.
- How deep it goes
- Explains repeated sampling and its role in posterior estimation, but does not detail specific algorithms or sampling mechanics.
- Examples
- Names genomics, climate science, financial modeling, and engineering design as application areas.
- What you can do after
- Understand how repeated sampling can estimate a posterior when exact solutions are impractical.
Read how Timeless Algorithms, The Foundational Ideas explains markov chain monte carlo →
What to know first
- posterior probabilityA posterior probability or distribution represents the probability of an event, hypothesis, or unknown quantity conditional on observed evidence and prior information.
- conditional probabilityConditional probability is the probability of an event or variable value given that specified information, an event, or another variable’s value is known.
- gibbs samplingGibbs sampling is a Markov chain Monte Carlo method that iteratively samples each variable from its conditional distribution given the current values of the other variables.
- Markov chainA Markov chain is a stochastic process of states whose transition probabilities depend on the current state rather than the full preceding history.
Related concepts
- metropolis-hastings samplingincludes · Metropolis-Hastings sampling is a Markov chain Monte Carlo method that proposes candidate states and accepts or rejects them with probabilities chosen to preserve a target distribution.
- variational inferencealternative to · Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
- datauses · Data is recorded or represented information, observations, measurements, or values that can be stored, processed, communicated, or interpreted.
- grid approximationalternative to · A grid approximation estimates a distribution by evaluating and normalizing probability weights on a finite grid of values.
- stochastic variational inferencealternative to · Stochastic variational inference optimizes a variational approximation using stochastic estimates of an objective or its gradient, often computed from data subsets.
- unnormalized posterioruses · A probability or distribution proportional to the posterior, given by the prior times the likelihood before normalization.
Questions about markov chain monte carlo
What is markov chain monte carlo?
Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution.
Which Manning books explain markov chain monte carlo?
Grokking Bayes, Machine Learning Algorithms in Depth, Causal AI and Timeless Algorithms, The Foundational Ideas.
What should I know before learning markov chain monte carlo?
Manning books that teach markov chain monte carlo usually introduce posterior probability, conditional probability, gibbs sampling and Markov chain first.
What is the difference between markov chain monte carlo and variational inference?
Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution. Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
What kinds of markov chain monte carlo are there?
Kinds of markov chain monte carlo include metropolis-hastings sampling.
What is markov chain monte carlo also called?
Markov chain monte carlo is also called MCMC.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



