Verbosity bias is the tendency to favor unnecessarily long responses that appear authoritative or helpful even when extra content is redundant or adds no value. It affects both language-model judges and agentic systems, where unnecessary steps or repeated tool calls can simulate progress. section 2.3.2 section 4.3.2
Diagnose and control verbosity bias by checking:
- Whether a longer answer adds useful information or merely repeats existing points. section 4.3.2
- Whether evaluation criteria explicitly reward focused, concise responses rather than exhaustive coverage. section 4.3.2
- Whether comparisons are length-normalized or penalize outputs that deviate sharply from a task’s reference length distribution. section 2.3.2 section 4.3.2
The repetitive-list attack
Responses from MT-Bench containing numbered lists were expanded by doubling list items with paraphrased synthetic content. Models preferred the longer lists despite their redundant information, isolating verbosity preference from genuine quality. section 4.3.2

