A baseline model is a deliberately simple predictive model that establishes a reference level of performance before more complex alternatives are tried. It is a measuring stick: if a later model does not improve meaningfully on it, added complexity may not be justified. section 7.2.3 section 12.1
For tabular regression, linear regression is a practical baseline; for classification, logistic regression is a practical counterpart. Their simplicity makes variable effects and failure patterns easier to inspect than in a more complex model. section 7.4 section 7.2.3
Use a baseline model to:
- Benchmark a complex model and test whether greater complexity adds value. section 7.2.3
- Detect possible overfitting when an advanced model performs worse than the baseline on unseen data. section 7.2.3
- Validate preprocessing, because simple models make variable effects less likely to be obscured. section 7.2.3
- Provide a minimum viable predictive solution while revealing underfitting, nonlinearities, interactions, and predictor characteristics. section 7.2.3 section 7.4
Airbnb price prediction baseline
After transforming the Airbnb Tokyo data into numeric features and handling missing values, fit a linear regression model to predict listing price. Generate cross-validation predictions, then calculate R-squared, RMSE, and MAE. The baseline produced R-squared 0.320, RMSE 17,197.323, and MAE 12,568.371; optimized XGBoost later improved these to 0.729, 10,853.661, and 6,611.609. section 7.2.3 section 12.1
Cross-validation metrics summarize performance across folds and estimate generalization. Cross-validation predictions instead supply an unbiased prediction for every training case from validation folds, enabling inspection of prediction ranges, extreme errors, and the coefficients contributing to individual cases. section 7.2.3
Listing 7.14 Linear regression baseline model with diagnostic plots
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_val_predict
from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
lm = LinearRegression(fit_intercept=False) #1
lm.fit(Xt, y) #2
cv_splits = cv.split(
X,
y = X["neighbourhood_more_than_30"]
) #3
y_pred_cv = cross_val_predict(
lm, Xt, y, cv=cv_splits
) #4
prediction_range = y_pred_cv.min()} - {y_pred_cv.max()
print(f"prediction range: {prediction_range}") #5
r2 = r2_score(y, y_pred_cv)
rmse = np.sqrt(mean_squared_error(y, y_pred_cv))
mae = mean_absolute_error(y, y_pred_cv) #6
print(f'R-squared: {r2:.3f}')
print(f'RMSE: {rmse:.3f}')
print(f'MAE: {mae:.3f}')
plt.scatter(y, y_pred_cv) #7
plt.plot([y.min(), y.max()], [y.min(), y.max()], 'r--', label='Ideal Fit')
plt.axhline(
0, color='orange', linestyle='--', label='Zero Line'
) #8
plt.xlabel('Actual')
plt.ylabel('Predicted')
plt.title('Linear Regression - Fitted Results')
plt.legend()
plt.show()
This runnable linear-regression workflow fits the baseline, creates cross-validation predictions, and calculates R-squared, RMSE, and MAE for diagnostic comparison. section 7.2.3
Treat baseline results as evidence for the next model rather than as a final verdict. Inspect where the simple model underfits or overfits, then use clues about nonlinearities, interactions, the target, and predictors to choose initial hyperparameter values for a complex model such as XGBoost. section 7.4 section 12.1
