Self-consistency tests whether independent full responses converge on the same substantive conclusion, even when their wording differs. Use it for analytical tasks with multiple valid reasoning paths and one shared conclusion to find. For bare labels, use mode; for a single numeric value, use mean. section 8.4
Run the comparison as a repeatable workflow:
- Generate independent responses to the same task. section 8.4.2
- Normalize each response into a short structured view, such as trigger, impact, and corrective action. section 8.4.2
- Identify shared conclusions and material divergences. If responses disagree materially, increase the sample size or escalate for analyst review rather than forcing agreement. section 8.4.2
Incident root-cause drafts
For an incident following checkout-api v2.18.4, compare independent drafts on the trigger, impact, and corrective action. The drafts can agree that deployment preceded degradation and rollback restored service while differing on whether database pressure caused the incident. Treat unsupported causal claims as divergences, not as established facts. section 8.4.1 section 8.4.2
| Field |
Generation 1 |
Generation 2 |
Generation 3 |
|---|---|---|---|
| Trigger |
Deployment of v2.18.4 |
Deployment caused degradation |
Rollout of v2.18.4 at 14:03 |
| Impact |
Error spike, DB connection pool saturation |
Error budget exceeded, DB pool saturation |
Error increase, DB pool saturation alerts |
| Causal framing |
Symptom stated, no causal link drawn |
Deployment caused resource pressure (inferred) |
Sequence suggests connection pressure (hedged) |
| Corrective action |
Rollback, stabilize on previous version |
Rollback, telemetry review, predeploy guard |
Rollback at 14:12, stabilize on previous build |
Compare three incident drafts by trigger, impact, causal framing, and corrective action to see where conclusions align or diverge. section 8.4.1
When complete responses do not converge and the goal is to select the strongest candidate against predefined criteria, best-of-N may fit better than seeking agreement. section 8.4.2
