liveBook · Manning

Posterior predictive check

As explained in Grokking Bayes

A posterior predictive check is a Bayesian model-checking technique that simulates data from posterior parameter uncertainty and compares the simulations with observed data.

A model can produce a plausible-looking posterior and still miss important patterns in the data. A posterior predictive check asks a practical question: if the fitted model generated another dataset like the one observed, would that dataset resemble what actually happened? It checks the model against the data used to fit it, rather than measuring how well it predicts future cases. section 4.0 section 4.5

The basic procedure is:

  • Draw values of the model’s unknown quantities from their posterior distributions. section 4.5
  • Use each draw to simulate possible observations, retaining the spread across simulations rather than making only one prediction. section 4.5 section 4.1
  • Compare the simulations with the observed data. Look for observations the model rarely generates or patterns it fails to reproduce. section 4.5 section 5.13

Tea or coffee?

Suppose three of five surveyed people prefer tea. Draw possible tea-preference probabilities from the posterior, then simulate the number of tea drinkers in groups of five. With a strongly tea-skeptical Beta(0.1, 10) prior, only 652 of 10,000 simulated groups match the observed count of three; fewer than 2% exceed it. The simulations tend to fall below the observation, revealing a problem that is easier to see in replicated counts than in parameter estimates alone. A Beta(1, 1) prior gives more reasonable posterior predictive samples in this comparison. section 4.6

House prices

For a house-price regression, simulate prices at the observed house sizes and plot each simulated distribution beside its actual price. Other checks include counting how often observed prices fall within approximate 95% predictive intervals and inspecting standardized residuals, or z-scores. These comparisons ask whether the model reproduces both the prices and their spread. section 4.7

Posterior predictive checks can expose a poor fit, but passing one does not prove a model correct. They complement prior predictive checks, which inspect assumptions before conditioning on data. When posterior samples come from MCMC, perform sampling diagnostics as well; a reassuring predictive plot does not replace checking the samples themselves. section 4.12 section 5.13

How Grokking Bayes covers it

How deep it goes
Works through posterior sampling, simulated counts, plots, predictive intervals, and z-scores; stops short of treating a passed check as proof.
Examples
Uses tea-versus-coffee counts and house-price predictions, with simulated distributions compared against observations.
What it assumes
Assumes a fitted prior-and-likelihood model with a posterior; introduces distribution sampling as a practical foundation for checking it.
What you can do after
Diagnose/debug with it by finding observed outcomes or patterns that posterior-generated datasets seldom reproduce.
How it connects
Links posterior distributions and likelihood-based simulation to prior predictive checks, Bayesian linear regression, and MCMC diagnostics.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.