Grid search evaluates every selected combination of parameter values, using cross-validation to identify a configuration that performs well. It is useful when parameters interact: a learning rate that works alone may behave differently with another number of estimators. section 4.4.2 section 4.3.2
A practical grid-search workflow is:
- Choose parameter grids, such as n_estimators values of 200–500 and learning_rate values of 0.6–1.0. section 4.4.2
- Select a scoring function appropriate to the task; balanced accuracy supports multiclass or imbalanced classification. section 4.4.2
- Run GridSearchCV with five folds, parallel processing, and refit=True so the best configuration is retrained on all training data. section 4.4.2
- Inspect the scores and parameters for every combination, then evaluate the selected estimator on held-out test data. section 4.4.2
AdaBoost digit classification
A grid containing four n_estimators values and three learning_rate values tests twelve AdaBoost configurations. The best setting was learning_rate=0.6 and n_estimators=200; the refitted model achieved 0.95 test accuracy on 360 digit examples. section 4.4.2
Exhaustive search is thorough but can be computationally expensive. RandomizedSearchCV tests a smaller sample of combinations and can be faster when the parameter space is large. section 4.4.2 section 7.5
