Grid approximation estimates an unknown posterior by laying a finite grid across the hypothesis space, evaluating the unnormalized posterior at each point, and normalizing those values into probabilities. section 5.5
Its usefulness depends mainly on dimensionality and where probability mass lies:
- It works well for one- or two-dimensional variables, but the required grid grows exponentially with dimensionality. section 5.5
- A three-point grid requires 3 points in one dimension, 9 in two, and 27 in three dimensions to maintain comparable coverage. section 5.5
- It wastes computation over low-density regions because it covers the entire hypothesis space instead of concentrating on likely values. section 5.5
Tea versus coffee
A discrete tea-versus-coffee model divided the range from 0 to 1 into an 11-hypothesis grid and normalized posterior values at those grid points. Extending the same resolution requires 121 hypotheses in two dimensions and 161,051 in five. section 5.5
Figure 5.2 We need exponentially more samples for a grid approximation as the dimensionality grows.
Figure 5.2 visualizes how maintaining comparable grid accuracy requires exponentially more samples as dimensionality increases. section 5.5
