liveBook · Manning

Markov chain

As explained in Grokking Bayes

A Markov chain is a stochastic process in which the distribution of the next state depends on the current state rather than the full preceding history.

A Markov chain is a random process whose next state depends only on its current state, not its complete history. section 5.5

In MCMC, the chain forms a dependent walk through a hypothesis space, spending more time in high-density regions so its samples approximate a target posterior. section 5.5 section 5.7

Two design choices shape this walk:

  • A proposal distribution usually suggests nearby points while occasionally encouraging exploration elsewhere. section 5.7
  • An accept/reject rule filters proposed moves so the chain follows the target distribution’s geography. section 5.7

A dependent sample chain

For a Beta(4,3) target, independent samples appear scattered across the interval, while MCMC samples arrive as an ordered chain whose points gradually fill out the distribution. section 5.5

Figure 5.3 Differences between i.i.d. sampling (left) and MCMC (right). MCMC generates a chain of dependent samples (order indicated by arrows) that approximates the distribution.

Compare independent samples with the ordered, dependent walk produced by MCMC. section 5.5

How Grokking Bayes covers it

How deep it goes
Defines state dependence and explains proposal and accept/reject mechanisms, but defers the mathematical proof of convergence.
Examples
Uses Beta(4,3) samples, ordered MCMC walks, grid approximations, and a tourist-attractions analogy.
What you can do after
Evaluate how dependent MCMC samples can still approximate a target posterior distribution.
How it connects
Connects Markov chains to MCMC, posterior distributions, i.i.d. sampling, proposal distributions, and accept/reject rules.

Also explained in

Compare how 7 books explain Markov chain →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.