A residual is an observed outcome minus the value predicted by a fitted model. It shows how far the model missed on one record. In regression, examining residuals alongside coefficients and overall fit metrics helps reveal discrepancies those summaries can hide. section 9.4 section 9.1.2
Use residuals to check where a regression model struggles:
- Plot residuals against predicted values. Look for patterns, changing spread, gaps, and unusually large errors rather than judging the model by one fit metric. section 9.2.2 section 4.3.2
- Inspect the underlying records behind outliers. A suspicious prediction may reflect a data problem; a legitimate outlier calls for a considered modeling decision. section 9.2.2
- Check the distribution of residuals and revisit model assumptions. Patterns may point to a missing variable or transformation; inconsistent error spread may call for a transformed outcome or weighted least-squares regression. section 9.4 section 4.3.2
housing_prices["residuals"] = model.resid
housing_prices["predicted_values"] = model.predict() #1
plt.scatter(
housing_prices["predicted_values"],
housing_prices["residuals"]
) #2
plt.axhline(y=0, color= "black", linestyle="-") #3
The housing-price workflow stores fitted residuals and predictions, then plots each residual against its predicted price with a zero line for reference. section 9.1.2
Investigating a rat-sighting prediction
A residual plot for daily rat sightings shows a gap and outliers among low predicted counts. Inspecting those records reveals one with missing weather data previously filled with zero. Removing that bad record and reassessing the plot is more defensible than automatically discarding every outlier; other unusual records may be legitimate. section 9.2.2
Treat a residual plot as a diagnostic, not a verdict. After correcting data or changing the outcome—for example, with a square-root transformation—fit the model again and check its residuals before relying on its predictions. section 9.2.2 section 9.4
