liveBook · Manning

Effective sample size

As explained in Machine Learning Algorithms in Depth

Effective sample size is a measure of the amount of independent information represented by a possibly weighted or correlated sample.

Effective sample size measures how much independent information a weighted or correlated sample contributes. For importance sampling, it compares the variance of a weighted average with that of an unweighted average: correlated samples have variance σ²/neff. section 2.7

Use effective sample size to assess importance-sampling quality:

  • Large, balanced importance weights indicate that the proposal resembles the target and produces lower estimator variance. section 2.7
  • A small effective sample size signals that a few weights dominate the weighted sum. section 2.7
  • Choose the acceptable effective-sample-size threshold according to the application. section 2.7
Figure 2.15 Importance sampler pseudo-code
figure

Importance-sampler pseudocode provides an implementation context for computing and using effective sample size as a diagnostic. section 2.7

How Machine Learning Algorithms in Depth covers it

How deep it goes
Derives effective sample size by equating weighted and unweighted-average variances, then applies it as a diagnostic.
Examples
Uses importance-sampler pseudocode and visualizations of sampling components and estimates.
What it assumes
Assumes familiarity with importance sampling, variance, weighted averages, and correlated samples.
What you can do after
Diagnose importance-sampling estimators by interpreting effective sample size and weight concentration.
How it connects
Connects importance sampling with importance weights, weighted averages, estimator variance, and correlated samples.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.