Occam’s razor favors the simplest hypothesis that explains the evidence. In data science, start with the least complex workable approach, then add complexity only when the problem requires it. section 3.1.6 section 4.3.2
Choosing a model for Airbnb prices
For Airbnb NYC price prediction, compare practical alternatives before reaching for deep learning. If linear regression solves the task, support vector machines are unnecessary; if conventional code solves it, machine learning may be unnecessary. In the comparison, XGBoost required fewer model-definition and training lines than Keras, making it the simpler tabular-data solution. section 3.1.6
Use the principle to make projects easier to deliver and operate:
- Prefer simpler implementations to obtain initial results faster, finish development sooner, and reduce maintenance after deployment. section 3.1.6
- For linear models, constrain complexity with regularization when added features and transformations risk overfitting. section 4.3.2
- Treat simplicity as problem-dependent: the principle favors simple linear models, whereas greater complexity can help neural networks on tabular data. section 4.3.2
Listing 4.5 L2 regularized linear regression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import Ridge, Lasso
column_transform = ColumnTransformer(
[('categories', categorical_onehot_encoding, low_card_categorical),
('numeric', numeric_passthrough, continuous)],
remainder='drop',
verbose_feature_names_out=False,
sparse_threshold=0.0)
polynomial_expansion = PolynomialFeatures(degree=2) #1
model = Ridge(alpha=2500.0) #2
model_pipeline = Pipeline(
[('processing', column_transform),
('polynomial_expansion', polynomial_expansion),
('standardizing', numeric_standardization),
('modeling', model)]
) #3
cv = KFold(5, shuffle=True, random_state=0)
rmse = make_scorer(mean_squared_error, squared=False)
cv_scores = cross_validate(estimator=model_pipeline,
X=data[price_window],
y=target_regression[price_window],
scoring=rmse,
cv=cv,
return_train_score=True,
return_estimator=True) #4
mean_cv = np.mean(cv_scores['test_score'])
std_cv = np.std(cv_scores['test_score'])
fit_time = np.mean(cv_scores['fit_time'])
score_time = np.mean(cv_scores['score_time'])
print(f"{mean_cv:0.3f} ({std_cv:0.3f})",
f"fit: {fit_time:0.2f} secs pred: {score_time:0.2f} secs") #5
Inspect a scikit-learn pipeline that combines polynomial expansion with L2-regularized Ridge regression, illustrating controlled complexity in a tabular workflow. section 4.3.2
