Agent collusion occurs when multiple agents reinforce incorrect conclusions because they share assumptions, biases, or training data, while no effective external check breaks the loop. section 7.1
A grounding agent can worsen the problem when it uses the same model as the evaluated agent, validating the same hallucinations instead of challenging them. section 7.1
Reduce collusion risk with layered checks:
- Use different models for evaluator and evaluated agents when practical, since different providers tend to fail in different directions. section 7.1
- When model diversity is impractical, use contrasting prompts or personas that make the evaluator search for problems rather than confirm correctness. section 7.1
- Review flagged outputs and random unflagged samples with humans, and log disagreements so automated evaluation can be audited. section 7.1
Figure 7.1 The external agent evaluation and feedback systems for a single user-facing agent. In the figure, a generative agent may face evaluation from a critic for general output feedback or from a grounding agent for context-specific feedback and evaluation.
Use the external evaluation diagram to distinguish critic feedback from context-specific grounding feedback around a user-facing agent. section 7.1
