liveBook · Manning

Grid approximation

As explained in Grokking Bayes

A grid approximation estimates a probability distribution by evaluating and normalizing weights at a finite set of points across a parameter space.

Grid approximation estimates an unknown posterior by laying a finite grid across the hypothesis space, evaluating the unnormalized posterior at each point, and normalizing those values into probabilities. section 5.5

Its usefulness depends mainly on dimensionality and where probability mass lies:

  • It works well for one- or two-dimensional variables, but the required grid grows exponentially with dimensionality. section 5.5
  • A three-point grid requires 3 points in one dimension, 9 in two, and 27 in three dimensions to maintain comparable coverage. section 5.5
  • It wastes computation over low-density regions because it covers the entire hypothesis space instead of concentrating on likely values. section 5.5

Tea versus coffee

A discrete tea-versus-coffee model divided the range from 0 to 1 into an 11-hypothesis grid and normalized posterior values at those grid points. Extending the same resolution requires 121 hypotheses in two dimensions and 161,051 in five. section 5.5

Figure 5.2 We need exponentially more samples for a grid approximation as the dimensionality grows.

Figure 5.2 visualizes how maintaining comparable grid accuracy requires exponentially more samples as dimensionality increases. section 5.5

How Grokking Bayes covers it

How deep it goes
Explains grid construction, posterior normalization, dimensional scaling, and inefficiency, but stops short of implementation details.
Examples
Uses a tea-versus-coffee model with an 11-point grid and compares grid sizes across one, two, and five dimensions.
What it assumes
Assumes familiarity with hypothesis spaces and unnormalized posteriors while framing the method through a discrete probability model.
What you can do after
Apply grid approximation by evaluating unnormalized posterior values on a low-dimensional grid and normalizing them.
How it connects
Connects posterior distributions, unnormalized posteriors, i.i.d. sampling, and Markov chain Monte Carlo.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.