A model can produce a plausible-looking posterior and still miss important patterns in the data. A posterior predictive check asks a practical question: if the fitted model generated another dataset like the one observed, would that dataset resemble what actually happened? It checks the model against the data used to fit it, rather than measuring how well it predicts future cases. section 4.0 section 4.5
The basic procedure is:
- Draw values of the model’s unknown quantities from their posterior distributions. section 4.5
- Use each draw to simulate possible observations, retaining the spread across simulations rather than making only one prediction. section 4.5 section 4.1
- Compare the simulations with the observed data. Look for observations the model rarely generates or patterns it fails to reproduce. section 4.5 section 5.13
Tea or coffee?
Suppose three of five surveyed people prefer tea. Draw possible tea-preference probabilities from the posterior, then simulate the number of tea drinkers in groups of five. With a strongly tea-skeptical Beta(0.1, 10) prior, only 652 of 10,000 simulated groups match the observed count of three; fewer than 2% exceed it. The simulations tend to fall below the observation, revealing a problem that is easier to see in replicated counts than in parameter estimates alone. A Beta(1, 1) prior gives more reasonable posterior predictive samples in this comparison. section 4.6
House prices
For a house-price regression, simulate prices at the observed house sizes and plot each simulated distribution beside its actual price. Other checks include counting how often observed prices fall within approximate 95% predictive intervals and inspecting standardized residuals, or z-scores. These comparisons ask whether the model reproduces both the prices and their spread. section 4.7
Posterior predictive checks can expose a poor fit, but passing one does not prove a model correct. They complement prior predictive checks, which inspect assumptions before conditioning on data. When posterior samples come from MCMC, perform sampling diagnostics as well; a reassuring predictive plot does not replace checking the samples themselves. section 4.12 section 5.13
