When a regression estimates shipping costs, RMSE answers a practical question: how far do predictions typically fall from actual costs, expressed in the same units as the cost itself? section 3.4.2
RMSE is the square root of mean squared error. Because it uses the dependent variable’s units, it is easier to interpret than MSE; smaller values indicate better predictive performance. Squaring errors before taking the root gives relatively greater influence to large mistakes. section 3.4.2 section 4.6
Use RMSE alongside other regression checks rather than treating it as a complete verdict:
- Compare RMSE values for competing models; the weight model’s RMSE of 12.52 is lower than the volume model’s 12.79. section 3.4.2
- Read RMSE in the target variable’s units: the shipping-cost models have typical error magnitudes of roughly 12.5 to 12.8 cost units. section 3.4.2
- Check MAE, MSE, R2, residuals, diagnostic plots, and model assumptions before relying on the estimate. section 3.4.2 section 4.6
Estimating freight-value error
For product-weight and product-volume regressions predicting freight value, the reported results are RMSE=12.52 and RMSE=12.79 respectively. Weight performs slightly better by this metric, but the R2 values—0.37 and 0.34—show that neither model captures all variation in shipping costs. section 3.4.2 section 4.6
Listing 3.4 Predicting product weight or volume influence on freight value
from sklearn.metrics import mean_absolute_error, mean_squared_error,
r2_score
# Split the data into features (X) and target (y)
X_weight = analysis_df["product_weight_g"].values.reshape(-1, 1)
X_weight = sm.add_constant(X_weight) # Add a constant for the intercept
X_volume = analysis_df["product_volume"].values.reshape(-1, 1)
X_volume = sm.add_constant(X_volume) # Add a constant for the intercept
y = analysis_df["freight_value"].values
# Predict the target variable using the models
y_pred_weight = model_weight.predict(X_weight)
y_pred_volume = model_volume.predict(X_volume)
# Calculate the evaluation metrics for both models
mae_weight = mean_absolute_error(y, y_pred_weight)
mse_weight = mean_squared_error(y, y_pred_weight)
rmse_weight = np.sqrt(mse_weight)
r2_weight = r2_score(y, y_pred_weight)
mae_volume = mean_absolute_error(y, y_pred_volume)
mse_volume = mean_squared_error(y, y_pred_volume)
rmse_volume = np.sqrt(mse_volume)
r2_volume = r2_score(y, y_pred_volume)
# Print the evaluation metrics
print(f"Product Weight Model: MAE={mae_weight:.2f}, MSE={mse_weight:.2f},
RMSE={rmse_weight:.2f}, R2={r2_weight:.2f}")
print(f"Product Volume Model: MAE={mae_volume:.2f}, MSE={mse_volume:.2f},
RMSE={rmse_volume:.2f}, R2={r2_volume:.2f}")
Inspect the Python workflow that calculates MAE, MSE, RMSE, and R2 from predictions produced by statsmodels regression models. section 3.4.2
