Multicollinearity occurs when predictors in a regression are strongly linearly related, inflating the variance of estimated coefficients and complicating interpretation. The variance inflation factor (VIF) measures this effect for each predictor. section 5.2.2
Use VIF to assess and respond to predictor redundancy:
- Regress one predictor against the remaining predictors, obtain its R², and use that value to calculate its VIF. section 5.2.2
- Treat VIF values of 1 or higher as evidence that multicollinearity is present; consider remediation when any VIF reaches 10 or higher. section 5.2.2
- Reduce the model by discarding offending predictors, while generally retaining the intercept even when its p-value is not significant. section 5.2.2
Checking a logistic model in Python
For a raisins classification model, regress MinorAxisLength on the other predictors with statsmodels, then read the resulting R². The value 0.9751 indicates that this predictor is highly explained by the others. A statsmodels VIF table reports extreme values, including Area at 404.77, MajorAxisLength at 129.29, and MinorAxisLength at 40.20. A reduced model using MinorAxisLength and Perimeter is then fitted for comparison.
