Chapter 9: Crimes against data: Statistics done wrong

Introduction

The chapter introduction explains how good intentions and flexible modeling can lead people to misuse statistics.

• Good intentions and statistical misuse: Explain how a desire to make an analysis work or troubleshoot it can lead to statistical misuse without malicious intent.
  Why it matters: Recognizing this risk helps you question an analysis even when the person doing it means well.

Key terms: wrong objective, flexibility that modeling can offer, misused

9.1 The importance of reading past headlines

This section shows why a striking statistical headline should be checked against the study details and limitations reported later in the article.

• Headlines versus study context: Explain why a reader should look beyond a news headline before accepting its statistical claim.
  Why it matters: Reading the details helps you avoid treating an attention-grabbing summary as the whole finding.
• Diet study claim limitations: Explain why the cited fasting-diet study did not establish that the eating window caused the reported heart-disease deaths.
  Why it matters: These limitations explain why an alarming association should not be read as proof that fasting caused the deaths.

Key terms: headlines, critical context, 16:8 diet, observational, self-reported, confounding variables

9.2 Confounding variables

This section explains how unseen variables can mislead interpretations of correlations and how to look for and limit their influence.

• Confounding and causal uncertainty: Explain what a confounding variable is and why a correlation, even with a low p-value, does not establish which variable causes the other.
  Why it matters: A convincing-looking correlation can still support the wrong causal story.
• Misleading clues and model predictions: Describe how the cookie and cow-image examples show an overlooked factor leading to a wrong conclusion or prediction.
  Why it matters: These examples make it easier to spot how a plausible interpretation can go wrong.
• Questioning and controlling alternatives: Describe how to investigate possible confounders and design a comparison that limits them, using the fasting study as an example.
  Why it matters: Questioning alternatives and controlling comparisons reduces the risk of attributing an outcome to the wrong cause.

Key terms: confounding variables, correlated, low p-value, confounding variable, computer vision system, human in the loop, controlled experiments

9.3 Data bias

This section explains how unrepresentative samples distort findings and why researchers should account for possible data bias.

• Unrepresentative samples and findings: Explain that data bias occurs when a sample fails to represent its population because relevant information was left out of selection or sampling, potentially exaggerating or downplaying findings. Use the college-student survey example to illustrate the mismatch.
  Why it matters: A finding about a population can be misleading if the sample leaves out people it is supposed to represent.
• Bias awareness and disclosure: Describe how recognizing possible bias can help researchers improve data collection, and why they should disclose biases they cannot adequately control.
  Why it matters: Acknowledging possible omissions helps keep claims about the data honest.

Key terms: data bias, data sample, population, selection, sampling process, bias, mitigate, disclose, integrity

9.4 Selection bias

This section explains how selection bias overrepresents some groups in a sample and how to recognize and reduce its effects during data collection.

• Selection bias in samples: Explain how selection bias arises when some groups are more likely than others to enter a sample, and how that can distort findings. Illustrate it with an example from the section.
  Why it matters: A result can be misleading when the people you hear from differ from the people you miss.
• Selection bias mitigation: Describe the questions and data-collection choices the section recommends for reducing or disclosing selection bias, and explain why self-selection bias is hard to eliminate entirely.
  Why it matters: Planning for who is included and excluded makes the limits of a finding clearer and can reduce avoidable bias.

Key terms: selection bias, sample, overrepresented, self-select, self-selection bias, randomly sample, population, mitigation strategies

9.5 Recall bias

This section explains how selective memory distorts reported data and why prompt written records help reduce recall bias.

• Selective memory in recalled data: Explain that recall bias occurs when people report what they remember vividly while omitting less memorable or unfavorable events, as in forgetting snacks when estimating a day’s calories.
  Why it matters: An honest answer can still misrepresent the data if important events are forgotten.
• Prompt records over memory: Describe why recall bias worsens when people must remember longer periods and how detailed records made soon after each event mitigate it.
  Why it matters: Prompt records make it less likely that a later account will omit events.
• Nostalgia in happiness polls: Explain how nostalgia can filter out negative memories of earlier decades and distort polls comparing past and present happiness.
  Why it matters: Comparisons of past and present happiness may reflect selective memories rather than the full experience of either period.

Key terms: recall bias, detailed records, nostalgia

9.6 Anchor bias

This section explains how early experiences or data can disproportionately shape judgments and systems, and how to recognize and limit that influence.

• First information as an anchor: Explain anchor bias as favoring the first information received, and describe how it can appear in people's preferences or a pricing system.
  Why it matters: Recognizing an anchor helps you question whether an apparent preference or a system's decision reflects current evidence.
• Reducing initial data weight: Describe ways to keep early data from dominating later data when it may not represent normal conditions.
  Why it matters: These choices can keep unusual early observations from outweighing more representative later ones.
• Preferences versus best judgments: Distinguish a poll about people's preferred programming language from a claim about which language is best, and describe how first-learned languages affect interpretation.
  Why it matters: A survey answer is useful only when interpreted in light of what the survey is trying to measure.

Key terms: anchor bias, first piece of information, older data points, outliers, sample, best, preferred, first language

9.7 Confirmation bias

This section explains how confirmation bias favors evidence that supports a desired conclusion, illustrates the pattern, and shows how it can narrow the information people encounter.

• Selective evidence and alternatives: Explain how confirmation bias affects information gathering and how to counter it.
  Why it matters: Considering unwelcome explanations helps keep a preferred conclusion from controlling which evidence counts.
• UFO story evidence dismissal: Trace how the UFO-channel narrator dismisses an alternative explanation and proceeds with a preferred one.
  Why it matters: The story makes it easier to recognize selective dismissal when a competing explanation is disappointing.
• Organizational and platform feedback: Describe how selective publication and personalized information can reinforce existing beliefs.
  Why it matters: Bias can shape which information is available, not just how one person interprets it.

Key terms: confirmation bias, contrary data, alternative explanations, alternative explanation, rationalize, echo chamber, feedback loop, homogeneous

9.8 Survivorship bias

This section explains survivorship bias through successful businesses and returning wartime bombers, showing how missing failures can reverse a conclusion.

• Missing failures in success samples: Explain why studying only successful businesses cannot establish which traits made them successful. Describe how failed businesses are both necessary for comparison and harder to observe.
  Why it matters: A trait common among winners may also be common among losers, so winners alone cannot show what separates them.
• Returning bomber damage inference: Trace why Abraham Wald recommended armoring areas without bullet holes on returned bombers rather than the visibly damaged areas.
  Why it matters: The bomber example shows how accounting for missing cases can change a practical decision.

Key terms: survivorship bias, surviving samples, failed businesses, class imbalance, returned planes, bullet holes, Abraham Wald

9.9 The operating domain

The section explains how defining a system’s operating domain guides data collection and reveals gaps that can make a sample unrepresentative or a model unreliable.

• Operating domain and scope: Explain that an operating domain is the environment and conditions in which a system must work, including conditions that change over time. Describe why a broader domain makes representative sampling harder.
  Why it matters: The conditions a model faces determine whether its sample can represent its real use.
• Domain-driven data requirements: Describe how a thorough definition of the operating domain guides the variables, sample size, and situations to collect, including possible oversampling of especially important situations.
  Why it matters: Defining the domain before collecting data helps expose situations a sample might otherwise miss.
• Consequences of missing conditions: Explain, using the section’s examples, why data from a limited set of conditions may not support performance in a wider domain.
  Why it matters: A model or test result can be unreliable precisely where the collected data do not cover its intended use.

Key terms: operational design domain (ODD), operating domain, representative sample, data requirements, oversampling, data coverage, extrapolating

9.10 Data coverage and the S-curve

The section explains why a model’s capability can level off as data and development costs rise, and why narrowing its operating domain makes adequate data coverage more achievable.

• Operating Domain and Data Coverage: Explain how the tasks and conditions assigned to a model determine the data it must cover, and why narrowing or controlling those conditions can reduce expense and improve its chances of success.
  Why it matters: The job you give a model determines how many situations its data must prepare it to handle.
• Capability S-Curve and Rescoping: Describe how capability initially improves with added data and expense but then levels off, and explain why a model that plateaus short of its target may need a narrower scope rather than simply more investment.
  Why it matters: Recognizing diminishing returns helps you decide when to narrow a model’s job instead of continuing to spend for smaller gains.

Key terms: operating domain, data coverage, outliers, edge cases, S-curve, inflection point, diminishing returns, rescope

9.11 Data drift

Explains how changing operating conditions make training data stale, degrade model performance, and require ongoing evaluation and updates.

• Data drift and model performance: Explain how changes in the environment that produces data can make training data stale and reduce a model’s performance, even if it worked well before.
  Why it matters: Good performance on familiar data does not guarantee that a model will keep working as conditions change.
• Limits of automatic adaptation: Explain why a model cannot always keep up with new trends automatically: new data may need human labeling, which can be slow and costly.
  Why it matters: Collecting fresh data does not necessarily mean a model can learn from it immediately.
• Planning for changing data: Describe the section’s response to possible data drift: plan for data’s shelf life, collect new data when conditions change, and update and evaluate the model using consistent metrics.
  Why it matters: A plan to monitor and refresh a model helps reveal whether it remains useful as its operating domain changes.

Key terms: data drift, data rot, operating domain, shelf life, manual labeling, annotated, human in the loop, quality assurance, monitoring, retrain

9.12 P-hacking

Defines p-hacking as selecting data or models for favorable results and explains how to recognize and mitigate the tendency.

• P-hacking and favorable results: Explain that p-hacking means cherry-picking data or models to obtain a desired result, often a low p-value or statistically significant finding, rather than a realistic one. Describe how it can happen unintentionally, even without deliberately calculating p-values.
  Why it matters: Recognizing p-hacking helps you question whether a striking result reflects the data or the search for a favorable outcome.
• Recognizing and mitigating p-hacking: Describe the section’s suggested response to possible p-hacking: notice when you are confirming your own biases, step away, and reconsider the problem with a willingness to abandon the approach.
  Why it matters: A willingness to reconsider an approach makes it less likely that a desired outcome will drive the analysis.

Key terms: P-hacking, low p-value, statistically significant finding, confirming your own biases, re-evaluate the problem

9.13 Texas sharpshooter fallacy

The section explains why finding an unlikely result after searching through data does not, by itself, make that result meaningful.

• Texas sharpshooter fallacy: Explain how drawing a target around a bullet hole illustrates the mistake of treating a result discovered after the fact as though it had been predicted. Describe how dredging large amounts of data can similarly turn chance observations, including low p-values, into apparently significant findings.
  Why it matters: It helps you question striking findings that were noticed only after someone searched through many possibilities.

Key terms: Texas sharpshooter fallacy, dredge data, low p-values

9.14 Torturing data and models

The section shows how flexible choices about data and models can manufacture desired results, and how fresh data and context-aware decisions help expose p-hacking.

• Desired results through flexible choices: Explain how p-hacking uses choices about data or models to support a desired finding rather than test it realistically, and give examples from the section.
  Why it matters: A result can look convincing because of the choices made to obtain it, not because it reflects the data fairly.
• Repeated testing and fresh validation: Trace how repeatedly changing a model or data to improve its test-dataset result can undermine that test, and describe the role of an additional validation dataset.
  Why it matters: A strong test result is less persuasive when repeated attempts were made to achieve it.
• Context for removing outliers: Distinguish a justified outlier removal from a misleading one using the section’s sleep-log examples.
  Why it matters: An inconvenient observation may be exactly the evidence the analysis needs to retain.
• Reproducibility and contrary evidence: Explain the section’s concern about findings that cannot be replicated and describe its suggested response to possible p-hacking.
  Why it matters: A finding is more credible when it can withstand evidence gathered to challenge it.

Key terms: p-hacking, cherry-picking, data transformations, outliers, train/test split, testing dataset, validation dataset, normal sleep habits, reproducibility crisis, replicate

9.15 Why people p-hack

This section explains why people p-hack under personal and institutional pressure, and how to counter it by seeking evidence that might challenge their findings.

• Pressures behind p-hacking: Explain how trying to get a model to work can slide into p-hacking, especially when research funding, job expectations, or startup investment depends on favorable results.
  Why it matters: Recognizing these pressures helps you notice when a desired outcome may be shaping an analysis.
• Startup demonstration gap: Describe why startups may cherry-pick favorable conditions, models, or datasets for investor demonstrations, and why a promising demonstration may not reflect real-world production.
  Why it matters: An impressive demonstration is less convincing when its conditions were selected to make the technology look good.
• Challenging favorable findings: Describe the section’s recommended response to p-hacking: notice when it is happening and gather new data that could challenge the findings.
  Why it matters: Looking for data that might undermine a result helps counter the pull of a desired conclusion.

Key terms: p-hacking, research pressure, job pressure, startup pressure, venture capital, cherry-pick, S-curve, real-world production, new data, challenge your findings

9.16 Summary