Hyperparameter tuning systematically searches tested parameter values, first locating the best order of magnitude and then refining values around the most promising settings. Keeping candidate lists short is important when several parameters must be evaluated in combination. section 18.2.4
A practical tuning protocol is:
- Choose coarse, heterogeneously spaced values for each parameter. section 18.2.4
- Measure results for the combinations of parameter values. section 18.2.4
- Repeat the search around the most promising values, using narrower intervals. section 18.2.4
Refining mutation chance
Begin with mutation chances such as [0.01, 0.05, 0.1, 0.2, 0.5, 0.8, 1]. If 0.2 and 0.5 perform best, test intermediate values such as 0.15, 0.2, 0.25, 0.3, through 0.6. section 18.2.4
Table 18.2 Average best solution for the TSP genetic algorithm, varying the operators’ chance. The results are sorted by tour cost, and we highlighted the most significant rows. (view table figure)
Compare average best TSP solutions while varying crossover, adjacent-swap, and random-swap chances. section 18.2.4
