liveBook · Manning

Sufficient statistic

As explained in Timeless Algorithms, The Foundational Ideas

A sufficient statistic is a statistic that contains all information in a sample relevant to a specified parameter under a specified model.

A statistic is sufficient for a parameter when it retains all information in the observed data relevant to that parameter under a specified probability model. After computing it, further analysis of the original observations adds no information for inference about that parameter. section 3.2.3

Sufficiency can make data reduction lossless for a particular inferential goal: a compact summary can replace the full sample without sacrificing information relevant to the parameter. Sufficiency depends on the statistical model, so a statistic may be sufficient under one model but not another. section 3.2.3

Estimating a normal mean

For observations 5, 7, 6, 8, and 4 from a normal distribution with known variance, the sample mean, 6, is sufficient for the population mean. The individual observations add no information about that mean beyond their average. section 3.2.3

Sufficiency is one of Fisher’s criteria for a good estimator, alongside consistency and efficiency. It concerns preserving information relevant to the parameter, not merely producing a compact summary. section 3.2 section 3.9

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines sufficiency, explains model dependence and lossless reduction, and works through a normal-mean example.
Examples
Uses five observations from a normal distribution with known variance to show the sample mean is sufficient for the population mean.
What you can do after
Evaluate whether a statistic preserves all sample information relevant to a parameter under a specified model.
How it connects
Connects sufficiency to consistency, efficiency, maximum likelihood estimation, and statistical models.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.