Monte Carlo Tree Search
Monte Carlo Tree Search is a decision-making algorithm that explores a search tree using repeated sampling or simulation to estimate the value of actions.
Applied Reinforcement Learning
Monte Carlo tree search is a planning algorithm that incrementally builds a search tree by selecting nodes, expanding actions, simulating outcomes, and backpropagating outcome statistics.
- How it is used
- Applies selective planning to business decisions with branching alternatives, especially capacity-constrained vehicle routing.
- How deep it goes
- Explains the four search phases and UCB1, then implements a routing solver with greedy rollouts and local improvement.
- Examples
- Uses delivery-fleet routing with customer demands, vehicle capacities, nearest-neighbor rollouts, and 2-opt route improvement.
- What you can do after
- Build it from scratch for capacitated vehicle routing using a tree of states, simulated completions, and reward updates.
Read how Applied Reinforcement Learning explains Monte Carlo Tree Search →
Grokking Bayes
Monte Carlo tree search is a sequential decision algorithm that simulates selected future branches and propagates their outcomes through a search tree.
- How it is used
- Guides sequential decisions by sampling strategically selected futures instead of averaging over every possible future.
- How deep it goes
- Explains selective branch exploration, simulation, and utility backpropagation, but does not specify a complete implementation procedure.
- Examples
- Uses board-game engines such as AlphaZero for chess and AlphaGo for Go as practical examples.
- What you can do after
- Evaluate/understand how sampled futures approximate expected-utility decisions in sequential problems.
Also covered in
- Designing AI AgentsIn early access (MEAP)
What to know first
- exploitationIn optimization and decision-making, exploitation is selecting an option currently believed, based on available information, to yield high value.
- explorationExploration is the selection of actions or approaches with uncertain outcomes to gain information or enable greater future value.
- Monte Carlo simulationMonte Carlo simulation is a computational technique that uses repeated random sampling to estimate quantities or characterize possible outcomes.
- multi-armed banditA multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
Related concepts
- backpropagation phasehas part · The backpropagation phase of tree search propagates an evaluated outcome or value from a searched node toward its ancestors, updating statistics along the path.
- dynamic programminguses · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
- game treeuses · A game tree is a branching representation whose nodes denote game states and whose edges denote possible moves between them.
- large language modeluses · A large language model is a machine-learning model trained on extensive text data to process and generate natural language.
- Markov decision processuses · A Markov decision process is a mathematical model of sequential decision-making defined by states, actions, probabilistic transitions, and rewards, with future dynamics depending on the current state and action.
- Monte Carlo simulationuses · Monte Carlo simulation is a computational technique that uses repeated random sampling to estimate quantities or characterize possible outcomes.
Questions about Monte Carlo Tree Search
What is Monte Carlo Tree Search?
Monte Carlo Tree Search is a decision-making algorithm that explores a search tree using repeated sampling or simulation to estimate the value of actions.
Which Manning books explain Monte Carlo Tree Search?
Applied Reinforcement Learning and Grokking Bayes.
What should I know before learning Monte Carlo Tree Search?
Manning books that teach Monte Carlo Tree Search usually introduce exploitation, exploration, Monte Carlo simulation and multi-armed bandit first.
What are the parts of Monte Carlo Tree Search?
Monte Carlo Tree Search includes backpropagation phase.
What is Monte Carlo Tree Search also called?
Monte Carlo Tree Search is also called MCTS.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

