Beam search
Beam search is a heuristic search or decoding technique that repeatedly expands a limited set of the highest-scoring partial candidates.
Natural Language Processing in Action, Second Edition
Beam search generates sequences by retaining a limited set of high-scoring partial paths and expanding them over multiple decoding steps.
- How it is used
- Guides generative language-model decoding toward high-scoring sequences without exploring every possible path.
- How deep it goes
- Explains retained paths, cumulative probability, breadth, depth, and recursive expansion, but does not provide beam-search implementation code.
- Examples
- Uses a road-trip analogy and a decision-tree figure showing two possible beams.
- What you can do after
- Apply beam search to bound generative decoding while retaining multiple candidate token sequences.
Read how Natural Language Processing in Action, Second Edition explains beam search →
Build an Advanced RAG Application (From Scratch)
Beam search is a sequence-decoding method that maintains multiple high-scoring partial hypotheses and expands them to find a strong overall sequence.
- How it is used
- Uses multiple candidate paths to generate coherent decoder-model text instead of committing to one token sequence immediately.
- How deep it goes
- Explains initialization, expansion, cumulative scoring, selection, and termination, but does not descend into model implementation internals.
- Examples
- Demonstrates two-token beam width through the expanding candidates “the quick brown” and “the cat sat.”
- What you can do after
- Apply beam search to generate sequences by expanding, scoring, pruning, and terminating candidate hypotheses.
Read how Build an Advanced RAG Application (From Scratch) explains beam search →
Transformers in Action
Beam search is a decoding algorithm that maintains a fixed number of highest-probability partial token sequences and expands each sequence at every step.
- How it is used
- Guides token-sequence generation as a deterministic alternative to greedy decoding and probabilistic sampling.
- How deep it goes
- Explains beam expansion and pruning using a search-tree analogy, then demonstrates generation parameters in a model call.
- Examples
- Illustrates a two-beam continuation from “LLMs” and configures five beams to return three sequences.
- What you can do after
- Apply beam search via model.generate by setting beam count, returned sequence count, and generation parameters.
Retrieval Augmented Generation, The Foundational Ideas
Beam search is a heuristic decoding algorithm that retains a fixed number of highest-scoring partial sequences at each step.
- How it is used
- Decodes generated sequences while avoiding locally probable choices that lead to globally weaker sentences.
- How deep it goes
- Explains candidate expansion, pruning, scoring, and runtime weighting, but does not develop beam-search implementation code.
- Examples
- Uses a city-and-state evidence-splitting example and compares document-level and token-level beam-search workloads.
- What you can do after
- Operate/configure decoding by selecting beam candidates and tuning reflection weights for fluency or factual grounding.
Read how Retrieval Augmented Generation, The Foundational Ideas explains beam search →
Optimization Algorithms: AI techniques for design, planning, and control problems
An informed search technique that retains only a limited number of the most promising partial solutions at each step.
- How it is used
- Finds short paths in graphs or trees when limiting memory and search breadth matters.
- How deep it goes
- Explains state retention, expansion, ranking, complexity, thresholds, and failure modes, stopping short of optimality guarantees.
- Examples
- Demonstrates S-to-G routing with w = 2 and larger routing experiments using k = 10 and n = 20.
- What you can do after
- Apply it via Python graph code by tuning beam width, generated children, costs, and route-expansion parameters.
Deep Learning with Python, Third Edition
A decoding method that keeps a fixed number of candidate generation paths and expands them step by step.
- How it is used
- Uses multiple candidate token chains to guide text-generation decoding.
- How deep it goes
- Defines fixed-width, stepwise exploration but stops short of implementation details or a worked beam-search example.
- What you can do after
- Evaluate beam search as a decoding alternative to single-path and sampling-based generation.
Read how Deep Learning with Python, Third Edition explains beam search →
Sutskever's List
Beam search is a sequence-decoding algorithm that retains a fixed-size set of high-scoring partial candidates at each expansion step.
- How it is used
- Turns probabilistic speech-model scores into a transcript while avoiding exhaustive sequence search.
- How deep it goes
- Explains candidate expansion, acoustic and language-model scoring, and pruning, without giving formal scoring equations.
- Examples
- Uses Deep Speech 2’s English and Mandarin beam sizes and Mandarin’s shared-sound search challenge.
- What you can do after
- Understand how retaining and pruning alternatives trades decoding cost against transcript quality.
Also covered in
- LLMs in Production
- Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
- Grokking AI Algorithms, Second Edition
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- tokenA token is a discrete unit of text, such as a word, subword, punctuation mark, or symbol, produced by tokenization and processed by a language model.
- softmaxSoftmax is a function that exponentiates each value in a score vector and normalizes the results to produce positive values summing to one, forming a probability distribution.
- attentionAttention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
Related concepts
- greedy algorithmalternative to · A greedy algorithm constructs a solution incrementally by choosing the locally best available option at each step, generally without reconsidering earlier choices or optimizing the entire sequence globally.
- informed searchkind of · Informed search is a search strategy that uses problem-specific knowledge or heuristic estimates to prioritize promising states or paths.
- decoding strategykind of · A decoding strategy is a procedure for converting an encoded or internal representation into a target output, including selecting tokens during sequence generation.
- self-ragused by · Self-RAG is a retrieval-augmented generation method in which a model uses its own outputs or self-assessment to guide retrieval and evaluate retrieved information or generated responses.
Questions about beam search
What is beam search?
Beam search is a heuristic search or decoding technique that repeatedly expands a limited set of the highest-scoring partial candidates.
Which Manning books explain beam search?
Natural Language Processing in Action, Second Edition, Build an Advanced RAG Application (From Scratch), Transformers in Action, Retrieval Augmented Generation, The Foundational Ideas, Optimization Algorithms: AI techniques for design, planning, and control problems, Deep Learning with Python, Third Edition and Sutskever's List.
What should I know before learning beam search?
Manning books that teach beam search usually introduce language model, token, softmax and attention first.
What is the difference between beam search and greedy algorithm?
Beam search is a heuristic search or decoding technique that repeatedly expands a limited set of the highest-scoring partial candidates. A greedy algorithm constructs a solution incrementally by choosing the locally best available option at each step, generally without reconsidering earlier choices or optimizing the entire sequence globally.
What is beam search a kind of?
Beam search is a kind of informed search. Informed search is a search strategy that uses problem-specific knowledge or heuristic estimates to prioritize promising states or paths.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






