Linear regression models a continuous outcome as a line-shaped relationship with one or more predictors. A simple model has the form y = mx + b: m describes the change in the modeled outcome for a one-unit change in x, and b is the intercept. With multiple predictors, each contributes a term to the estimated outcome, such as y = β₀ + β₁X₁ + β₂X₂. section 4.3.2 section 9.1.1
Choose it when the question calls for estimating a value, such as a house price, rather than a category. Pearson’s correlation can help identify an initial linear association, but regression combines predictors into an equation that can estimate outcomes for new inputs. A strong correlation is a reason to investigate a model, not proof that the model’s predictions or causal interpretation will hold. section 4.3 section 9.1.1 section 9.1.2
Predicting house prices
An ordinary least squares model can use square footage and number of bedrooms to estimate price. Its fitted coefficients describe each predictor’s contribution to the estimate; entering a new home’s values produces an estimated price. The estimate should be checked against observed prices rather than treated as an exact value. section 9.1.2
import statsmodels.api as sm
import pandas as pd
housing_prices = pd.read_csv("housing_prices.csv") #1
X = housing_prices[["sq_footage", "n_bedrooms"]]
y = housing_prices["price"] #2
X = sm.add_constant(X) #3
model = sm.OLS(y, X).fit() #4
print(model.summary())
The listing separates the continuous outcome from its predictors, adds a constant for the intercept, fits a statsmodels OLS model, and prints its diagnostic summary. section 9.1.2
Before relying on the fitted line:
- Examine scatterplots and correlations to see whether relationships look linear. If they do not, consider an appropriate transformation or another model. Avoid treating highly correlated inputs, such as daily high and low temperatures, as independent evidence without examining them. section 4.3.2 section 3.2.2 section 9.2.3
- Check whether the observations meet the model’s independence assumption. Inspect residuals—the differences between observed and predicted values—for patterns and extreme values. section 4.3.2 section 9.1.2 section 9.1.3
- Read coefficients alongside fit and complexity measures. R² summarizes the proportion of outcome variation explained by the inputs; adjusted R² penalizes adding predictors. Compare iterations rather than assuming that more inputs make a better model. section 9.4 section 9.2.3
For stakeholders, distinguish prediction from causation. Show important relationships clearly, explain what the model estimates, and state what its results cannot establish. For example, weather correlations can motivate a model of daily rat sightings for planning, but they do not by themselves show that weather causes the sightings. section 4.3.3 section 9.2
