In Monte Carlo tree search, selection follows the existing tree from the root, choosing a branch at each node. It directs search toward promising routes without ignoring under-explored ones. section 7.3
At each step, selection uses UCB1 to balance:
- Exploitation: favor nodes with higher average reward from past visits. section 7.3
- Exploration: give less-visited nodes a larger bonus. section 7.3
- Continue until reaching a node with unexplored actions or a terminal state. section 7.5
Choosing between routes
With the given visit counts and rewards, Route A scores −14.28 and Route B scores −45.36 under UCB1. Since selection chooses the higher score, it follows Route A. section 7.3
Figure 7.5 UCB1 algorithm for node selection in MCTS.
The UCB1 figure illustrates the score used to select a child node. section 7.3
