An outlier is a data point that sits far from the overall pattern of the remaining observations. It may reflect a measurement error, data-entry mistake, or rare event. Because a few extreme values can overwhelm the rest of a small data set, they can inject bias into a linear regression model. section 5.4
Start by looking at the distribution rather than trusting a single summary number. Outliers can pull the mean away from the median: career win shares for first-round NBA picks range from negative values to more than 236, and a small group of stars makes the mean substantially larger than the median. section 2.5.1 section 2.5.3
Use plots to spot unusually distant values:
- Try a scatterplot or histogram to see the shape and tails of a continuous variable, but recognize that deciding whether tail values are outliers can be subjective. section 5.4.1
- Use a boxplot when a consistent visual rule is needed: points beyond the whiskers are classified as outliers. section 5.4.1 section 14.5.1
- Compare groups with boxplots to see whether distributions, medians, and extreme observations differ across categories. section 2.5.3 section 14.5.1
Capping extreme deflections
For the deflections variable, scatterplots, a histogram, and a boxplot flag high values for review. Instead of deleting records from a short data set, winsorization replaces values above 17.8 with 17.8. A follow-up boxplot then shows no points beyond the whiskers. This preserves the rows while reducing the influence of the extremes. section 5.4.1 section 5.4
sp1 <- qplot(seq_along(hustle$deflections), hustle$deflections) +
labs(title = "Deflections",
subtitle = "scatterplot",
x = "",
y = "Value") +
theme(plot.title = element_text(face = "bold")) +
annotate("text", x = 65, y = 18.5,
label = "Outlier?", color = "red",
size = 3, fontface = "bold") +
annotate("text", x = 85, y = 18.3,
label = "Outlier?", color = "red",
size = 3, fontface = "bold")
hist1 <- ggplot(hustle, aes(x = deflections)) +
geom_histogram(fill = "snow1", color = "dodgerblue4", bins = 8) +
labs(title ="Deflections",
subtitle = "histogram",
x = "",
y = "Frequency") +
theme(plot.title = element_text(face = "bold")) +
annotate("text", x = 18.75, y = 3,
label = " Outliers?", color = "red",
size = 3, fontface = "bold")
bp1 <- ggplot(hustle, aes(x = "", y = deflections)) +
labs(title = "Deflections",
subtitle = "boxplot",
x = "",
y = "") +
geom_boxplot(color = "dodgerblue4", fill = "snow1", width = 0.5) +
stat_summary(fun = mean, geom = "point", shape = 20, size = 8,
color = "dodgerblue4", fill = "dodgerblue4") +
annotate("text", x = "", y = 18.6,
label = " Outliers",
color = "red", size = 3, fontface = "bold") +
theme(plot.title = element_text(face = "bold"))
Compare three views of deflections: the scatterplot and histogram suggest possible extremes, while the boxplot isolates observations beyond its whiskers. section 5.4.1
