Grid search trains a model with different combinations of specified hyperparameters, then selects the combination that performs best on a validation set. section 13.5
For an SVM with an RBF kernel, a small search can proceed like this:
- Choose candidate values for gamma and C; pairing two values for each gives four models to compare. section 13.5
- Train each model, compare its validation performance using a chosen metric, and keep the best-performing combination. section 13.5
Comparing SVM settings
In one comparison, gamma = 0.1 and C = 10 achieved the highest validation accuracy among the tested combinations: 0.72. section 13.5
svm_parameters = {'kernel': ['rbf'],
'C': [0.01, 0.1, 1 , 10, 100],
'gamma': [0.01, 0.1, 1, 10, 100]
} #1
svm = SVC() #2
svm_gs = GridSearchCV(estimator = svm,
param_grid = svm_parameters) #3
svm_gs.fit(features_train, labels_train) #4
This Scikit-learn example defines candidate values for C and gamma, creates a GridSearchCV object, and fits it to the training data. section 13.5
