Multicollinearity occurs when input features depend strongly on one another and carry nearly the same information. For example, patient weight and blood pressure may be highly correlated, allowing a simpler model to retain only one feature. section 7.1.1
A correlated-feature regression experiment
Start with one noisy feature, then construct a second feature as three times the first plus random noise. After a 75/25 train-test split, compare OLS, Ridge, LASSO, and Elastic Net regression. OLS becomes unstable with highly correlated features; Ridge improves robustness, while LASSO and Elastic Net can suppress one feature and produce sparse models. section 7.1.1
X = np.concatenate([X, 3*X + 0.25*np.random.uniform(size=(n, 1))], axis=1)
Inspect the construction of a two-feature data set in which the second feature depends on the first. section 7.1.1
