An i.i.d. sample has two properties: each observation is unrelated to the others, and every observation comes from the same probability distribution. These properties make sample-based estimates of means, credible intervals, and predictive quantities converge toward their true values as the sample grows. section 5.5
I.i.d. samples are the ideal for Monte Carlo estimation, but drawing them is usually impossible when a Bayesian posterior is known only up to a constant. Grid approximation can help in one or two dimensions, while Markov chain Monte Carlo uses dependent samples to approximate the target in higher-dimensional problems. section 5.14 section 5.5
Figure 5.3 Differences between i.i.d. sampling (left) and MCMC (right). MCMC generates a chain of dependent samples (order indicated by arrows) that approximates the distribution.
Compare independent samples with the dependent chain produced by MCMC, including how sample order differs between the two approaches. section 5.5




