liveBook · Manning

Best-of-N

As explained in Prompt Engineering in Practice

Best-of-N is a sampling technique that generates multiple complete candidate responses and selects the strongest candidate.

Best-of-N generates multiple complete responses to the same prompt, then selects one against predefined criteria. Use it when candidate quality differences matter and one final response is needed—not an average or vote. section 8.5 section 8.7

Run the selection workflow in this order:

  • Define the scoring criteria before reading any candidates to reduce anchoring on the first response. section 8.5
  • Generate N complete candidates from the same prompt and requirements. section 8.5.1
  • Score each candidate against the criteria, select one final response, and archive the rubric for auditability. section 8.5.1
  • Choose another technique when the output is a label, a single numeric score, or a low-consequence draft that will be heavily edited. section 8.5

Executive deployment update

For a failed production checkout deployment, generate three complete updates under the same prompt. The prompt sets a 120–160-word limit, targets senior engineering leadership, calls for a calm and action-oriented tone, and requires impact, mitigation, and next-update timing. Score the candidates against predefined criteria before choosing the final update. section 8.5.1

How Prompt Engineering in Practice covers it

How deep it goes
Generates complete candidates under identical requirements, scores them against predefined criteria, selects one, and archives the rubric; it does not average or vote.
Examples
Uses three candidate executive updates about a failed checkout deployment, constrained by length, audience, tone, and required content.
What it assumes
Requires criteria to be defined before candidates are read and keeps all candidates under the same requirements.
What you can do after
Evaluate complete candidates against predefined criteria and select one final response.
How it connects
Distinguishes best-of-N from mean, mode, and self-consistency by selecting a complete response rather than aggregating numbers, labels, or convergent outputs.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Prompt Engineering in Practice. The section links point to the parts of the book it draws on.