liveBook · Manning

Estimator

As explained in Timeless Algorithms, The Foundational Ideas

An estimator is a rule or statistic that maps observed data to an estimate of an unknown parameter.

An estimator is a rule or statistic that maps observed data to an estimate of an unknown parameter. Its quality depends on how it behaves as an estimate, not merely on producing a value. section 3.2

Three criteria characterize a good estimator:

  • Consistency: the estimate converges toward the true parameter as the amount of data increases. section 3.2
  • Efficiency: the estimator makes effective use of available information; for unbiased estimators, efficiency means attaining the smallest possible variance. section 3.2 section 3.9
  • Sufficiency: the estimator retains all sample information relevant to the parameter, allowing data reduction without sacrificing inferential accuracy. section 3.2 section 3.9

Maximum likelihood estimation (MLE) specifies a probability model and chooses parameter values that maximize the likelihood of the observed data. Under appropriate regularity conditions, it can yield estimators with desirable properties. Method of moments offers another approach, while Bayesian inference provides a distinct framework. section 3.2.4 section 3.9

Regression

Logistic regression estimates coefficients by maximizing the likelihood of observed binary outcomes. Linear regression with normally distributed errors illustrates a case where maximizing likelihood is equivalent to minimizing the sum of squared errors. section 3.9

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains the three criteria and relates them to maximum likelihood under appropriate regularity conditions.
Examples
Uses logistic regression coefficients and linear regression with normally distributed errors to illustrate maximum likelihood estimation.
What you can do after
Evaluate estimators by checking convergence, information use, and retention of parameter-relevant sample information.
How it connects
Connects estimators to maximum likelihood, method of moments, Bayesian inference, and statistical inference.

Also explained in

Compare how 5 books explain estimator →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.