liveBook · Manning

Verbosity bias

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Verbosity bias is a tendency for a language model to produce unnecessarily long responses that may appear more authoritative without adding useful information.

Verbosity bias is the tendency to favor unnecessarily long responses that appear authoritative or helpful even when extra content is redundant or adds no value. It affects both language-model judges and agentic systems, where unnecessary steps or repeated tool calls can simulate progress. section 2.3.2 section 4.3.2

Diagnose and control verbosity bias by checking:

  • Whether a longer answer adds useful information or merely repeats existing points. section 4.3.2
  • Whether evaluation criteria explicitly reward focused, concise responses rather than exhaustive coverage. section 4.3.2
  • Whether comparisons are length-normalized or penalize outputs that deviate sharply from a task’s reference length distribution. section 2.3.2 section 4.3.2

The repetitive-list attack

Responses from MT-Bench containing numbered lists were expanded by doubling list items with paraphrased synthetic content. Models preferred the longer lists despite their redundant information, isolating verbosity preference from genuine quality. section 4.3.2

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines the bias, demonstrates it experimentally, explains training causes, and proposes length normalization without detailing a complete implementation.
Examples
Uses the MT-Bench repetitive-list attack and unnecessary agent steps or looping tool calls as concrete examples.
What it assumes
Assumes familiarity with model evaluation and introduces the distinction between helpful elaboration and valueless padding.
What you can do after
Evaluate verbosity by length-normalizing comparisons and penalizing redundant or inefficient outputs.
How it connects
Connects verbosity bias to LLM-as-a-judge, BLEU, brevity penalty, reinforcement learning from human feedback, reward models, and evaluation rubrics.

Also explained in

Compare how 2 books explain verbosity bias →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.