Selection bias

As explained in Deep Learning with Python, Third Edition

A bias introduced when the data collection process produces a sample that is not representative of the target population.

Selection bias, also called sampling bias, appears when the data-collection process produces measurements that do not represent the target population. The problem is especially dangerous when who gets sampled is related to what you are trying to predict. section 6.1.2

The 1948 election poll

A phone survey suggested that Dewey would win the 1948 US presidential election, prompting the Chicago Tribune’s “DEWEY DEFEATS TRUMAN” headline. Phone users were not a random sample: they were more likely to be richer, conservative, and likely to vote for Dewey. The survey therefore measured a systematically skewed slice of voters. section 6.1.2

Practical rules of thumb:

• Check whether the collection process changes who appears in the dataset or how outcomes are measured. section 6.1.2
• Treat predictions from past data cautiously when the sample or the underlying behavior may change. section 6.1.2

“DEWEY DEFEATS TRUMAN”: a famous example of sampling bias

Use the historical polling example to see how an apparently ordinary collection method can produce a systematically misleading sample. section 6.1.2

How Deep Learning with Python, Third Edition covers it

How deep it goes

Defines the mechanism and works through a historical polling example, but does not detail correction methods.

Examples

Uses the 1948 US election phone survey and the “DEWEY DEFEATS TRUMAN” headline.

What you can do after

Evaluate whether a collection process produces a representative sample before trusting its predictions.

How it connects

Connects sampling bias with data collection, machine learning, prediction, and concept drift.

liveBookmanning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Deep Learning with R, Third EditionSampling bias is systematic distortion in measurements caused by a data-collection process that selects examples in relation to the quantity being predicted.
• 

  liveBookmanning.com
  Grokking StatisticsSelection bias is systematic distortion caused by including some groups in a sample more readily than others, making those groups overrepresented.
• 

  liveBookmanning.com
  Experimentation for EngineersSystematic measurement error caused by collecting samples under conditions that differ across the populations or variants being compared.

Compare how 4 books explain selection bias →

Related concepts

• biaskind of · Bias is a systematic deviation or tendency in data, measurements, estimates, judgments, or outputs that skews them relative to a relevant target or impartial baseline.
• samplingproduced by · Sampling is a procedure for selecting or generating observations, values, or outcomes from a larger population, signal, or probability distribution according to a specified method.
• randomizationcontrasts with · Randomization is the deliberate use of chance to select, assign, arrange, or vary elements of a process.
• bayesian statisticsuses · Bayesian statistics is a framework for statistical inference that combines prior probability distributions with observed-data likelihoods to update probabilities over hypotheses or unknown quantities.
• calibration datacontrasts with · Calibration data is data used to measure, estimate, or adjust a model’s parameters, outputs, or behavior for a specified calibration objective.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.