Anomaly

As explained in Statistics Every Programmer Needs

An outlier is an observation that is unusually distant from the pattern represented by the rest of a dataset.

An outlier is a data point that differs significantly from other observations. In linear regression, checking for outliers is part of assessing the data and the model’s fit. section 4.1.3 section 4.3

When assessing outliers in regression:

• Use a scatterplot as one possible detection method, while recognizing that it may not be accurate. section 4.1.3
• Interpret transformations cautiously: they change the scale and the relationship described by the fitted model. section 4.1.3

A regression fit after removal

Removing one presumed outlier from eight points and refitting the regression changed the coefficient of determination from 0.88 to 0.95. section 4.1.3

Figure 4.4 A scatter plot that displays seven data points rather than eight. The one presumed outlier was removed, and another regression was fit. The coefficient of determination, which was previously equal to 0.88, is now 0.95. By removing one outlier, we’ve made a strong model even stronger.

The figure shows the presumed outlier removed and the regression refitted. section 4.1.3

How Statistics Every Programmer Needs covers it

How deep it goes

Defines outliers, mentions scatterplot detection, and illustrates removal without detailing a detection procedure.

Examples

Removes a presumed point from eight observations and reports R² rising from 0.88 to 0.95.

What you can do after

Diagnose regression data by checking for outliers and evaluating their effect on model fit.

How it connects

Links outliers to linear regression, normality, and exploratory scatterplots.

liveBookmanning.com

Read Statistics Every Programmer Needs on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Outlier Detection in PythonAn extreme value is an observation far from the typical level of a time series, either unusually small or unusually large.
• 

  liveBookmanning.com
  Statistics Slam DunkAn outlier is an observation that lies unusually far from the other observations in a distribution or dataset.
• 

  liveBookmanning.com
  The Well-Grounded Data AnalystAn outlier is an observation whose value differs substantially from the typical pattern of the other observations.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIAn extreme value is an observation or value located unusually far toward an edge of an observed distribution.
• 

  liveBookmanning.com
  Data Without LabelsIn DBSCAN, an outlier or noise point is an observation that is neither a core point nor reachable from a core point and therefore receives no cluster assignment.
• 

  liveBookmanning.com
  Starting Data Analytics with Generative AI and PythonAn outlier is an observation that is unusually distant from the other values in a dataset or from an expected pattern.
• 

  liveBookmanning.com
  Cyber Threat HuntingAn outlier is an observation that lies unusually far from the central or typical range of a data set.
• 

  liveBookmanning.com
  Pandas Workout: 200 exercises to make you a stronger data analystAn outlier is an observation unusually far below or above the central range of a dataset according to a chosen criterion.
• 

  liveBookmanning.com
  Experimentation for EngineersAn observed data point that lies unusually far from the other observations in a dataset or fitted relationship.
• 

  liveBookmanning.com
  Data-Driven CybersecurityAn outlier is an observation that lies unusually far from the expected distribution or reference pattern.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentAn outlier is a data point whose behavior or value is anomalous relative to the dominant patterns in a dataset.
• 

  liveBookmanning.com
  Financial AI in PracticeAn outlier is a data value that is unusually distant from other observations in a dataset or distribution.
• 

  liveBookmanning.com
  Grokking StatisticsAn outlier is an observation that differs markedly from most other observations in a dataset.

Compare how 14 books explain anomaly →

Related concepts

• out-of-distributionincludes · Out-of-distribution describes data or inputs whose generating distribution differs materially from the distribution represented in a model’s training data.
• local outlierincludes · A local outlier is a data item that differs from other items in its nearby or comparable cluster while belonging to a common overall type.
• winsorizationaddressed by · Winsorization is a data transformation that replaces extreme observations beyond selected thresholds with less-extreme boundary values rather than removing them.
• interquartile rangeuses · The interquartile range is the difference between the third and first quartiles of a distribution, measuring the spread of its middle half.
• box-and-whisker plotused by · A box-and-whisker plot is a statistical graphic that summarizes a numerical distribution using quartiles, a median, and whiskers or markers indicating spread and potential outliers.
• contextual anomalyincludes · A contextual anomaly is an observation that is anomalous relative to its surrounding temporal, situational, or other contextual conditions.

This explainer was generated by Manning's knowledge graph from the text of Statistics Every Programmer Needs. The section links point to the parts of the book it draws on.