liveBook · Manning

Simulation phase

As explained in Applied Reinforcement Learning

The simulation phase is the Monte Carlo tree search step that rolls out actions from an expanded node to obtain an outcome estimate.

In Monte Carlo tree search, simulation completes a partial solution from an expanded node using a rollout policy, often random or greedy. The resulting outcome estimate helps compare paths without requiring perfect planning. section 7.5

Figure 7.4 The four-phase cycle of Monte Carlo Tree Search.

See how simulation fits into the repeating four-phase MCTS cycle. section 7.3

How Applied Reinforcement Learning covers it

How deep it goes
Completes a partial solution from an expanded node using a random or greedy rollout, but does not require perfect planning.
Examples
Uses partial vehicle-routing solutions and random or greedy policies to complete rollouts.
What you can do after
Evaluate partial solutions by rolling them out to obtain an outcome estimate.
How it connects
Connects Monte Carlo tree search with rollout policies and simulation-based evaluation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.