liveBook · Manning

Backpropagation phase

As explained in Applied Reinforcement Learning

The backpropagation phase of Monte Carlo tree search propagates a simulation outcome toward the root while updating node statistics.

In Monte Carlo tree search, backpropagation carries a simulation’s outcome from an expanded node toward the root. Updating statistics along the path lets later iterations learn from the rollout. section 7.5

For each node on the path from the expanded node to the root:

Figure 7.4 The four-phase cycle of Monte Carlo Tree Search.

See how backpropagation fits into the repeating four-phase MCTS cycle. section 7.3

How Applied Reinforcement Learning covers it

How deep it goes
Specifies a node-to-root pass that increments visits and adds simulation reward, without detailing reward design.
Examples
Uses a vehicle-routing search and a node-update pseudocode loop as concrete illustrations.
What you can do after
Apply the update loop by incrementing visits and accumulating rollout reward along the path to the root.
How it connects
Links Monte Carlo tree search with simulation rollouts, tree nodes, visit counts, and accumulated rewards.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.