liveBook · Manning

Greedy algorithm

Also called greedy heuristic, greedy strategy, greedy search

A greedy algorithm constructs a solution incrementally by choosing the locally best available option at each step, generally without reconsidering earlier choices or optimizing the entire sequence globally.

How 16 Manning books explain it

Algorithms Every Programmer Should Know

A greedy algorithm chooses a locally optimal option at each step without accounting for the effects of future choices.

How it is used
Tests whether immediate item selection can solve capacity-constrained optimization before turning to dynamic programming for indivisible items.
How deep it goes
Defines local selection and works through failure and success cases, but does not provide a greedy implementation.
Examples
Uses a 10 kg packing choice, an eight-credit cloud-instance budget, and closest-node route selection.
What you can do after
Evaluate/understand when greedy item selection fails because items are indivisible and when fractional selection makes it optimal.

Read how Algorithms Every Programmer Should Know explains greedy algorithm →

Optimization Algorithms: AI techniques for design, planning, and control problems

Greedy decoding constructs an output sequence by selecting the locally highest-scoring available choice at each decoding step.

How it is used
Builds solutions through repeated local choices, from cargo-bike loading to model-based TSP decoding.
How deep it goes
Works through a constrained loading example and model configuration, but does not establish optimality guarantees.
Examples
Uses profit-per-kilogram cargo loading and greedy decoding of a pretrained model on a TSP instance.
What you can do after
Operate a pretrained PyTorch model by configuring greedy decoding for TSP inference.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains greedy algorithm →

Build an Advanced RAG Application (From Scratch)

Greedy decoding is an autoregressive text-generation method that selects the highest-probability next token at each step without considering alternative continuations.

How it is used
Uses next-token probabilities to produce efficient, deterministic autoregressive text generation.
How deep it goes
Explains the selection loop, advantages, and limitations, but stops short of implementation code or model internals.
Examples
Demonstrates next-token selection with the fragment “The cat sat on the...” and candidate words such as “jumps,” “runs,” and “walks.”
What you can do after
Apply greedy decoding to generate text one token at a time using maximum-probability selection and explicit stopping conditions.

Read how Build an Advanced RAG Application (From Scratch) explains greedy algorithm →

Advanced Algorithms and Data Structures

A solution method that repeatedly makes the locally best available choice according to a specified ranking or criterion.

How it is used
Uses greedy choices to frame optimization problems such as packing goods and finding shortest routes.
How deep it goes
Defines local choice and illustrates where it succeeds or fails, without giving a general proof technique.
Examples
Compares food packing by calories or calories per kilogram with shortest-route selection and local optimization.
What you can do after
Evaluate whether a locally best choice fits a problem’s constraints before relying on a greedy solution.

Read how Advanced Algorithms and Data Structures explains greedy algorithm →

Build AI Drug Discovery Pipelines

Greedy decoding generates a sequence by selecting the highest-probability token at every step.

How it is used
Uses greedy token selection to generate SMILES sequences during inference without ground-truth characters.
How deep it goes
Defines highest-probability token selection and notes speed and sequence-quality tradeoffs without detailing its implementation.
Examples
Contrasts greedy decoding with temperature sampling when generating SMILES from a latent vector.
What you can do after
Evaluate generation strategies by recognizing that greedy decoding is fast but may produce suboptimal sequences.

Read how Build AI Drug Discovery Pipelines explains greedy algorithm →

Natural Language Processing in Action, Second Edition

Greedy search generates a sequence by selecting the highest-probability next token at each step without planning over alternative continuations.

How it is used
Uses greedy search to generate GPT-2 text by repeatedly selecting the next token.
How deep it goes
Defines one-step token selection and shows probability ranking, then notes repetition loops without deriving the algorithm formally.
Examples
Ranks “ new” and “ non” after “NLP is a” and shows repeated data-structure text.
What you can do after
Apply Hugging Face GPT-2 generation and adjust temperature or repetition penalties to change token selection behavior.

Read how Natural Language Processing in Action, Second Edition explains greedy algorithm →

Grokking Algorithms, Second Edition

A greedy algorithm constructs a solution by repeatedly making locally best choices without generally reconsidering earlier choices.

How it is used
Builds solutions through locally best choices and provides fast approximations when exact solutions take too long.
How deep it goes
Defines local and global choices, walks through successful and failed examples, and gives an O(n²) set-covering implementation.
Examples
Uses classroom scheduling and a 35-pound knapsack to show when greedy choices succeed or miss the optimum.
What you can do after
Evaluate whether a local-choice strategy finds an exact answer or a fast approximation, and implement repeated best-coverage picks.

Read how Grokking Algorithms, Second Edition explains greedy algorithm →

Effective Conversational AI

Greedy decoding is a language-generation strategy that selects the highest-scoring next token at each step.

How it is used
Supports repeatable LLM generation for intent-data augmentation, conversation summaries, and debugging.
How deep it goes
Defines next-token selection as choosing the most probable word and notes repeatability, without detailing decoding internals.
Examples
Illustrates password-phrase generation, repeatable conversation summaries, and recipe-response debugging.
What you can do after
Operate generation with greedy decoding to obtain repeatable outputs for a given input.

Read how Effective Conversational AI explains greedy algorithm →

Deep Learning with Python, Third Edition

A decoding strategy that selects the most likely predicted token at each step without considering future alternatives.

How it is used
Selects the next token during text generation by choosing the model’s most likely prediction at each step.
How deep it goes
Defines the strategy and demonstrates its implementation, while noting that it ignores future alternatives.
Examples
Uses an argmax function as the greedy strategy passed into a text-generation loop.
What you can do after
Apply it via Keras operations by selecting the highest-scoring token with argmax in a generation loop.

Read how Deep Learning with Python, Third Edition explains greedy algorithm →

Transformers in Action

Greedy decoding generates a sequence by selecting the highest-probability next token at each step without retaining alternative continuations.

How it is used
Controls token selection during text generation by choosing the most probable next token at each step.
How deep it goes
Defines stepwise conditional-probability selection and explains that locally likely choices can miss coherent or globally optimal continuations.
Examples
Illustrates greedy choices with a maze analogy and a sentence-completion prompt generated with sampling disabled.
What you can do after
Apply greedy decoding with Hugging Face model.generate by setting do_sample=False and a maximum token limit.

Read how Transformers in Action explains greedy algorithm →

Machine Learning Algorithms in Depth

A greedy algorithm repeatedly selects a locally optimal option in the hope of obtaining a globally optimal solution.

How it is used
Frames greedy choice as an algorithmic paradigm for optimization tasks, including knapsack selection and sensor placement.
How deep it goes
Defines local-choice heuristics, implements fractional knapsack, and links submodularity to guarantees without deriving a general proof.
Examples
Works through fractional knapsack and describes sensor placement using incremental mutual information.
What you can do after
Build a fractional-knapsack solver by ranking items by value-to-weight ratio and taking fractions when capacity runs short.

Read how Machine Learning Algorithms in Depth explains greedy algorithm →

Build a Large Language Model (From Scratch)

A decoding strategy that always selects the highest-probability next token at each generation step.

How it is used
Uses next-token inference as a deterministic baseline for generating text with a GPT model.
How deep it goes
Explains the iterative PyTorch loop, tensor positions, context truncation, softmax, argmax, and concatenation.
Examples
Demonstrates “Hello, I am” generation through encoded IDs, six iterations, decoded output, and an untrained model’s gibberish.
What you can do after
Apply it via PyTorch to generate and decode a fixed number of tokens from an encoded context.

Read how Build a Large Language Model (From Scratch) explains greedy algorithm →

Deep Learning with R, Third Edition

Greedy search is a sequence-generation strategy that selects the most probable next token at every generation step.

How it is used
Uses a per-step highest-probability token choice as a baseline strategy for generating text.
How deep it goes
Defines per-step maximum-probability selection and implements it with argmax, briefly contrasting random sampling.
Examples
Shows `greedy_search()` using `op_argmax` and notes that an early-trained model may repeat “the.”
What you can do after
Apply it via `op_argmax` as the sampling function passed to `compiled_generate()`.

Read how Deep Learning with R, Third Edition explains greedy algorithm →

Fabulous Adventures in Data Structures and Algorithms

A greedy algorithm constructs a solution by repeatedly choosing the locally best available option without generally reconsidering earlier choices.

How it is used
Uses greedy choices to reformat source code quickly when an acceptable result matters more than finding an optimal one.
How deep it goes
Defines subproblem optimization and demonstrates its trade-off with successful and failing change-making cases.
Examples
Uses US coin change for 73 cents and 41 cents without nickels, alongside source-code pretty printing.
What you can do after
Evaluate whether a fast greedy solution is acceptable by comparing it with a slower optimal alternative.

Read how Fabulous Adventures in Data Structures and Algorithms explains greedy algorithm →

Retrieval Augmented Generation, The Foundational Ideas

Greedy decoding generates a sequence by selecting the single most probable next token at each step.

How it is used
Serves as a standard decoding baseline for generating text one token at a time.
How deep it goes
Defines next-token selection and commitment, then contrasts this local choice with beam search's multi-candidate search.
Examples
Illustrates commitment through choosing the most probable next word and contrasts it with keeping the top-k continuations.
What you can do after
Evaluate/understand how greedy decoding commits to locally probable tokens, unlike beam search's retained alternatives.

Read how Retrieval Augmented Generation, The Foundational Ideas explains greedy algorithm →

Build a Reasoning Model (From Scratch)

A decoding strategy that always selects the highest-probability next token instead of sampling randomly.

How it is used
Uses highest-scoring token selection to implement sequential text generation during LLM inference.
How deep it goes
Explains score extraction, argmax indexing, shape preservation, decoding, and iterative reuse, while deferring alternative decoding strategies.
Examples
Demonstrates torch.argmax on score vectors and decodes token ID 20286 into “Large.”
What you can do after
Apply greedy decoding with PyTorch to select and decode each next token in an autoregressive generation loop.

Read how Build a Reasoning Model (From Scratch) explains greedy algorithm →

Also covered in

What to know first

Related concepts

Questions about greedy algorithm

What is greedy algorithm?

A greedy algorithm constructs a solution incrementally by choosing the locally best available option at each step, generally without reconsidering earlier choices or optimizing the entire sequence globally.

Which Manning books explain greedy algorithm?

Algorithms Every Programmer Should Know, Optimization Algorithms: AI techniques for design, planning, and control problems, Build an Advanced RAG Application (From Scratch), Advanced Algorithms and Data Structures, Build AI Drug Discovery Pipelines, Natural Language Processing in Action, Second Edition, Grokking Algorithms, Second Edition, Effective Conversational AI and 8 more.

What should I know before learning greedy algorithm?

Manning books that teach greedy algorithm usually introduce language model, token, accuracy and logit first.

What is the difference between greedy algorithm and beam search?

A greedy algorithm constructs a solution incrementally by choosing the locally best available option at each step, generally without reconsidering earlier choices or optimizing the entire sequence globally. Beam search is a heuristic search or decoding technique that repeatedly expands a limited set of the highest-scoring partial candidates.

What is greedy algorithm a kind of?

Greedy algorithm is a kind of algorithm. An algorithm is a finite, precisely specified procedure for performing a computation, transforming inputs into outputs, or solving a problem.

What is greedy algorithm also called?

Greedy algorithm is also called greedy heuristic, greedy strategy or greedy search.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.