Markov chain monte carlo
Markov chain Monte Carlo is a family of sampling algorithms that generate samples using a Markov chain whose stationary or limiting distribution is a target probability distribution.
Machine Learning Algorithms in Depth
Markov chain Monte Carlo is a Monte Carlo methodology that samples a Markov chain to approximate a target probability distribution, typically a posterior distribution.
- How it is used
- Approximates high-dimensional Bayesian posterior distributions through samples from a Markov chain.
- How deep it goes
- Defines the stationary-distribution principle and sketches Gibbs and Metropolis–Hastings updates without specifying a complete sampler implementation.
- Examples
- Uses inverse-CDF exponential sampling, π estimation, a binomial tree model, and a self-avoiding random walk as sampling examples.
- What you can do after
- Evaluate/understand it as a way to estimate posterior quantities and distinguish sampling from variational approximation.
Read how Machine Learning Algorithms in Depth explains markov chain monte carlo →
Causal AI
Markov chain Monte Carlo is a family of sampling algorithms that generate a Markov chain whose limiting distribution is a target probability distribution.
- How it is used
- Supports probabilistic inference after causal-model transformations, including interventions, conditioning, and counterfactual queries.
- How deep it goes
- Explains accept/reject chains and limiting convergence, then applies MCMC within a causal inference workflow.
- Examples
- Uses an online gaming query involving intervention E and outcome IE to demonstrate sampling from causal distributions.
- What you can do after
- Apply it via Pyro or PyMC to sample from distributions defined by transformed causal models.
Related concepts
- metropolis-hastings samplingincludes · Metropolis-Hastings sampling is a Markov chain Monte Carlo method that proposes state changes and accepts or rejects them probabilistically to generate samples from a target distribution.
- stochastic variational inferencealternative to · Stochastic variational inference is a variational inference technique that optimizes an objective using stochastic estimates, often computed from randomly sampled data subsets.
- variational inferencealternative to · Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
- posterior probabilityuses · A posterior probability or distribution expresses the probability assigned to an event, hypothesis, or unknown quantity conditional on observed evidence and prior information.
- Markov chainuses · A Markov chain is a stochastic process of states whose transition probabilities depend on the current state rather than the full preceding history.
- gibbs samplingincludes · Gibbs sampling is a Markov chain Monte Carlo technique that repeatedly samples each variable from its conditional distribution given the others.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.