An evaluation dataset is held-out data used to assess how well a predictive model performs on examples it did not use for fitting. In the example, an 80/20 split reserves the test set for this check. section 9.3.2
A practical evaluation workflow:
- Fit the model on training data, then predict outcomes for the held-out set. section 9.3.2
- Compare predictions with actual values; RMSE and R-squared are used to assess performance. section 9.3.2
- Compare training and test performance to check whether accuracy carries over to unseen data. section 9.3.2
from sklearn.metrics import mean_squared_error #1
y_pred_train = model.predict(X_train)
y_pred = model.predict(X_test) #2
r2_train = model.score(X_train, y_train)
r2_test = model.score(X_test, y_test) #3
rmse_train = np.sqrt(mean_squared_error(y_train, y_pred_train))
rmse_test = np.sqrt(mean_squared_error(y_test, y_pred)) #4
print(f"Training Set RMSE: {rmse_train}")
print(f"Test Set RMSE: {rmse_test}")
print(f"Training Set R-squared: {r2_train}")
print(f"Test Set R-squared: {r2_test}")
The example calculates RMSE and R-squared for both training and test sets. section 9.3.2
