liveBook · Manning

Multinomial

As explained in LLMs in Production

Multinomial sampling selects a categorical outcome randomly according to its probability distribution.

Multinomial sampling chooses the next token randomly from the model’s probability distribution rather than always selecting the highest-probability token. In generation, this can encourage more exploration and creativity than argmax, while temperature, top_k, and top_p constrain the candidates. section 9.1.2

Comparing decoding choices

Generate several continuations from the same prompt, then compare top_k versus top_p with multinomial sampling and argmax versus multinomial sampling. The exercise makes the trade-off between deterministic selection and exploratory output concrete. section 9.1.2

How LLMs in Production covers it

How deep it goes
Explains the generation workflow and sampling controls, but stops short of detailing multinomial’s underlying implementation.
Examples
Compares top_k and top_p with multinomial sampling, and compares multinomial sampling with argmax on generated text.
What it assumes
Assumes tokenized prompts and model logits, while introducing temperature scaling, probability filtering, softmax, and next-token selection.
What you can do after
Apply multinomial sampling in a token-generation loop and evaluate its output against argmax decoding.
How it connects
Connects multinomial sampling with logits, softmax, temperature, argmax, top-k sampling, top-p sampling, tokenization, and inference.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.