liveBook · Manning

Greedy algorithm

As explained in Deep Learning with R, Third Edition

Greedy search is a sequence-generation strategy that selects the most probable next token at every generation step.

Greedy search generates text by choosing the most probable next token at each step. This simple strategy can get stuck in loops of high-probability token sequences. section 16.2.4

greedy_search <- function(preds) {
  op_argmax(preds)
}

compiled_generate(prompt, greedy_search)

The `greedy_search()` function selects the highest-scoring prediction with `op_argmax` and passes that choice into text generation. section 16.2.4

How Deep Learning with R, Third Edition covers it

How deep it goes
Defines per-step maximum-probability selection and implements it with argmax, briefly contrasting random sampling.
Examples
Shows `greedy_search()` using `op_argmax` and notes that an early-trained model may repeat “the.”
What it assumes
Assumes next-token predictions from a language model and a generation loop that selects successive tokens.
What you can do after
Apply it via `op_argmax` as the sampling function passed to `compiled_generate()`.
How it connects
Connects token selection to language modeling, Transformers, random sampling, and top-k sampling.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 12 books explain greedy algorithm →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.