chapter eight
8 Prompt sampling
This chapter covers
- Recognizing that a single response is one probabilistic sample and that multiple independent generations often improve reliability
- Applying four sampling techniques (mean, mode, self-consistency, and best-of-N) by output type
- Reading spread and vote distribution as confidence signals to support or escalate decisions
- Knowing when to sample and when a single generation is enough
Chapters 2 through 7 establish how to shape what a language model produces in one generation: structural elements, linguistic elements, prompt patterns, prompt templates, prompt types, and contextual prompting.
This chapter addresses a different problem. Even with a strong prompt, one generation can still vary from the next when temperature is above zero (or when other sampling parameters such as top_p and top_k introduce variation). Prompt sampling improves reliability by collecting multiple independent generations and applying an aggregation or selection method.