Prompt sensitivity is the tendency for model outputs or measured performance to change when wording or formatting changes, even if the task’s meaning stays the same. section 6.3.2 section 6.3.3
Surface variations that can affect results include:
- Label formatting, separators, example order, and layout. section 6.3.2
- Tone and politeness, even when questions and answer options remain unchanged. section 6.3.2
- Equivalent prompt formulations whose effects differ across models. section 6.3.3
Tone can shift measured accuracy
In one multiple-choice study using ChatGPT-4o, accuracy rose from 80.8% for very polite prompts to 84.8% for very rude prompts. This result illustrates a measured difference, not a general rule that rudeness improves performance. section 6.3.2
Sensitivity varies with the model and the measurement method; some apparent effects may reflect rigid scoring. Prompt revisions without objective feedback can seem beneficial while leaving accuracy unchanged or reducing it. section 6.3.3 section 6.3.3
