Linear regression predicts a numeric target by adding weighted input features. After numeric conversion, including one-hot encoding for categories, each feature receives a coefficient; multiplying feature values by those coefficients and summing them produces a prediction. An intercept, or bias, can supply a baseline prediction, and coefficients may be constrained to positive values. section 4.3.1
The model selects coefficients that make predictions approximate observed targets by minimizing residual sum of squares. Its simple weighted-sum form makes feature contributions inspectable: coefficients can help explain a prediction to stakeholders, test it against domain expectations, and support regulatory justification. section 4.3.1
Prepare data carefully before fitting a linear regression model:
- Choose a continuous numeric target, such as listing price; a median split or decile bins instead turn that target into a classification problem. section 4.2
- Impute missing values, because the model cannot multiply or add missing feature values. section 4.3.1
- Investigate outliers, which can distort both fitting and predictions. section 4.3.1
- Check whether features and target are sufficiently linearly related; weakly related features can add noise, contributing to underfitting or, in large numbers, overfitting. section 4.3.1
Predicting Airbnb listing price
Treat price as the continuous target and transform listing attributes into numeric features. A pipeline can one-hot encode low-cardinality categories, pass continuous variables through, fit scikit-learn's LinearRegression, and estimate error with shuffled five-fold cross-validation using RMSE. Customized preparation can add indicators, log-transform skewed counts, and create interaction features such as Manhattan combined with entire-home status. section 4.2 section 4.3.1
Listing 4.3 Linear regression
from sklearn.linear_model import LinearRegression
from sklearn.metrics import make_scorer, mean_squared_error
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_validate
column_transform = ColumnTransformer(
[('categories', categorical_onehot_encoding, low_card_categorical),
('numeric', numeric_passthrough, continuous)],
remainder='drop',
verbose_feature_names_out=False,
sparse_threshold=0.0) #1
model = LinearRegression() #2
model_pipeline = Pipeline(
[('processing', column_transform),
('modeling', model)] #3
)
cv = KFold(5, shuffle=True, random_state=0) #4
rmse = make_scorer(mean_squared_error,
squared=False) #5
cv_scores = cross_validate(estimator=model_pipeline,
X=data[price_window],
y=target_regression[price_window],
scoring=rmse,
cv=cv,
return_train_score=True,
return_estimator=True) #6
mean_cv = np.mean(cv_scores['test_score'])
std_cv = np.std(cv_scores['test_score'])
fit_time = np.mean(cv_scores['fit_time'])
score_time = np.mean(cv_scores['score_time'])
print(f"{mean_cv:0.3f} ({std_cv:0.3f})",
f"fit: {fit_time:0.2f}",
f"secs pred: {score_time:0.2f} secs") #7
Use the runnable pipeline as a template for combining categorical encoding, numeric features, LinearRegression, five-fold validation, and RMSE scoring. section 4.3.1
A linear model is often a strong first baseline for tabular prediction: it creates a minimum viable solution, exposes preprocessing problems, and provides a transparent benchmark before adding complexity. Compare it on unseen validation folds with later models rather than assuming a more complex method improves the result. section 4.0 section 7.2.3
Inspect predictions as well as aggregate metrics. Cross-validation predictions can reveal implausible ranges, including negative price estimates, while per-case coefficient contributions show which transformed features pushed a particular prediction up or down. Such diagnostics indicate underfitting, nonlinearities, interactions, or unsuitable predictors that may motivate a more complex model. section 7.2.3 section 7.4
Baseline versus optimized gradient boosting
For an Airbnb regression task, a linear-regression baseline reached R-squared 0.320, RMSE 17,197.323, and MAE 12,568.371; optimized XGBoost reached 0.729, 10,853.661, and 6,611.609. The comparison illustrates why a transparent baseline is useful even when a nonlinear model ultimately performs better. section 12.1
