Argmax

As explained in LLMs in Production

Argmax is the operation that returns the position of the maximum value in a set or vector.

Argmax returns the position of the largest value in a set or vector. During language-model generation, applying it to token logits selects the token with the highest logit as the next token. section 9.1.2 section 7.2.2

Argmax provides a direct alternative to multinomial sampling: it selects the highest-scoring option rather than sampling among possibilities. Temperature zero is handled by using argmax, not by dividing logits by zero. section 9.1.2 section 7.2.2

Figure 7.3 A simple path of how the next word is chosen. Given an input, a model will generate a vector of logits for each token in the model’s vocabulary. Using the softmax algorithm, these logits will be transformed into probabilities. These probabilities will correspond to how often that token is likely to be chosen. Temperature is applied during the softmax algorithm.

Trace how a prompt produces token logits and probabilities before a next-token choice. section 7.2.2

How LLMs in Production covers it

How deep it goes

Explains highest-logit selection and its role in temperature handling, without detailing the algorithm's implementation.

Examples

Uses the prompt “I am a” to illustrate token logits and probabilities, and contrasts argmax with multinomial sampling.

What you can do after

Apply argmax to select the highest-logit token as the next token instead of sampling among alternatives.

How it connects

Connects argmax with logits, softmax, temperature, and multinomial sampling.

liveBookmanning.com

Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Learn Generative AI with PyTorch

Compare how 2 books explain argmax →

Related concepts

• greedy algorithmequivalent to · A greedy algorithm builds a solution incrementally by choosing the locally best available option at each step, without generally accounting for all future consequences or revising prior choices.
• top-k samplingequivalent to · Top-k sampling is a decoding technique that restricts each next-token selection to the k candidates with the highest probabilities, then samples from those candidates.
• generationused by · Generation is the computational production of outputs from supplied inputs or context, often by applying a model to produce predictions or new content.
• best-of-Nused by · Best-of-N is a technique that generates or evaluates multiple candidates and selects one according to a scoring or quality criterion.

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.