Estimator

As explained in Machine Learning Algorithms in Depth

An estimator is a rule that computes an approximation to an unknown statistical quantity from observed data or samples.

An estimator is a rule that uses observed samples to approximate an unknown statistical quantity. In importance sampling, weighted samples from a proposal distribution estimate expectations under a target distribution, even when normalization constants are unknown. section 2.7

Importance-sampling estimators are characterized by:

• Weights formed from target-to-proposal density ratios, including ratios of unnormalized densities. section 2.7
• Unbiasedness in theory under independent samples and consistency when importance-weight variance is finite. section 2.7
• Variance and convergence assessed through the estimator’s weight behavior and Chebyshev’s inequality. section 2.7

Estimating an expectation

An importance sampler draws positive samples from a truncated Gaussian proposal, evaluates target and proposal densities, computes their ratio as a weight, and estimates the expectation of f(x)=2 sin((π/1.5)x). Numerical integration supplies a ground-truth comparison. section 2.7

Figure 2.15 Importance sampler pseudo-code

Importance-sampler pseudocode makes the estimator’s sampling, weighting, and averaging procedure concrete. section 2.7

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives importance-sampling estimators, unbiasedness, variance, and consistency, stopping short of broader estimator theory.

Examples

Uses truncated Gaussian samples, density-ratio weights, a sinusoidal function, pseudocode, and numerical integration.

What it assumes

Assumes probability-density notation, sample independence for unbiasedness, and finite importance-weight variance for consistency.

What you can do after

Apply importance sampling to estimate expectations from normalized or unnormalized target and proposal distributions.

How it connects

Connects importance sampling with proposal distributions, normalization constants, the weak law of large numbers, and Chebyshev’s inequality.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA scikit-learn-compatible object that implements model fitting and typically prediction or transformation operations.
• 

  liveBookmanning.com
  Causal AIAn estimator is a statistic or rule that produces an estimate of an unknown population quantity.

Compare how 3 books explain estimator →

Related concepts

• point estimateproduces · A point estimate is a single numerical value used to estimate an unknown quantity or parameter.
• double machine learningused by · Double machine learning is a causal-effect estimation method that uses machine-learning nuisance models with orthogonalized estimation to reduce bias from nuisance-function errors.
• Scikit-Learn pipelinepart of · A scikit-learn pipeline is a composite estimator that sequentially chains named data transformations with a final estimator for fitting and prediction.
• fit methoduses · A fit method is an operation that learns or estimates model parameters, structure, or transformation statistics from supplied data.
• front-door estimandmeasures · A front-door estimand is a causal-effect quantity identified from observational data through a mediator that satisfies the front-door identification conditions.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.