Training data bias occurs when a sample fails to represent its population, causing analyses or models to produce distorted or inaccurate findings. Missing information often enters through the way data is selected or sampled, rather than through the model itself. For example, surveying college students and treating them as representative of all U.S. adults creates a misleading result. section 9.3
Common warning signs include:
- Selection bias: a group self-selects into the sample and becomes overrepresented, such as an airline polling only its own passengers. section 9.3
- Recall bias: people remember vivid or favorable events while forgetting less memorable ones, such as omitting a breakroom slice of pizza from a calorie log. section 9.3
- Anchor bias: the first information received dominates later evidence, such as an algorithm continuing to rely on unusually high launch-day demand. section 9.3
A broader check than “does the model work?”
Before sampling, consider the entire operating domain: the environments, conditions, and populations the model must handle. Include conditions that change over time, because omissions can degrade performance and amplify harmful outcomes in machine learning. section 9.16 section 1.5
Table 9.1 Some data bias types
| Bias type |
What it is |
Example |
How to avoid it |
|---|---|---|---|
| Selection bias |
A group self-selects into the sample and becomes overrepresented. |
An airline polling its own passengers about their favorite airline |
Randomly sample participants, and ask who is likely or unlikely to participate. |
| Recall bias |
Only remembering data that is more vivid or favorable, while forgetting less memorable or unfavorable data |
Tracking daily calories by memory and forgetting the slice of pizza grabbed from the breakroom |
Record data as quickly as possible after events; never rely solely on memory for data collection. |
| Anchor bias |
Overrelying on the first piece of information received at the expense of subsequent data |
A pricing algorithm anchored to a product’s high launch-day demand and failing to adjust as sales normalize |
Let older data points expire or be down-weighted; treat initial data points as potential outliers. |
| Confirmation bias |
Only gathering data that aligns with a desired outcome and dismissing contrary evidence |
A company promotes studies that are favorable to its product and buries ones that |
Actively seek alternative explanations; stop and reconsider when you notice you are dismissing contrary data. |
| Survivorship bias |
Only analyzing surviving (visible) samples and ignoring those that failed or never made it to the sample at all |
Books about the traits of successful companies and entrepreneurs that ignore the failures that share those same traits |
Actively seek out failed cases for comparison; ask why observed subjects survived, not just what they have in common. |
Use the bias table to compare each bias type with its example and a practical control, such as random sampling, prompt recording, or down-weighting older data. section 9.3
Mitigate bias by sampling randomly, recording events close to when they occur, and checking who is likely to participate. Disclose remaining limitations. Also seek contrary findings rather than repeatedly modeling data until a preferred result appears; that practice risks p-hacking and confirmation bias. section 9.3 section 9.16
