The advocate-strength effect occurs when a debate-style judge favors the response defended by the stronger advocate, making advocacy skill look like response quality. A single debate cannot separate these effects from position bias. section 10.3.3
Test whether a verdict survives changes to the evaluation setup:
- Swap advocate roles so each model argues for each response. section 10.3.3
- Swap the order in which the judge sees the arguments. section 10.3.3
- Aggregate the runs, report agreement frequency, and send low-agreement cases to human review. section 10.3.3
Assignment reversal
If Response A wins when model_X advocates for it but Response B wins after the assignments reverse, the result indicates advocate strength rather than a reliable quality difference. section 10.3.3
