A Markov chain is a random process whose next state depends only on its current state, not its complete history. section 5.5
In MCMC, the chain forms a dependent walk through a hypothesis space, spending more time in high-density regions so its samples approximate a target posterior. section 5.5 section 5.7
Two design choices shape this walk:
- A proposal distribution usually suggests nearby points while occasionally encouraging exploration elsewhere. section 5.7
- An accept/reject rule filters proposed moves so the chain follows the target distribution’s geography. section 5.7
A dependent sample chain
For a Beta(4,3) target, independent samples appear scattered across the interval, while MCMC samples arrive as an ordered chain whose points gradually fill out the distribution. section 5.5
Figure 5.3 Differences between i.i.d. sampling (left) and MCMC (right). MCMC generates a chain of dependent samples (order indicated by arrows) that approximates the distribution.
Compare independent samples with the ordered, dependent walk produced by MCMC. section 5.5






