Consistency means giving equivalent cases the same answer. Generative AI can vary its conclusions because it predicts tokens rather than following fixed rules; wording, chat history, task length, provider updates, or new training data can contribute to variation. In consequential settings, inconsistent answers create complaints, weaken trust, and complicate audits. section 6.1.3 section 6.1
ClaimAssist courtesy-car eligibility
Two customers with similar accidents ask whether their policies include a courtesy car. ClaimAssist gives one a clear yes and the other a no, despite accessing the same documents and both policies including the benefit. The team routes the decision through a deterministic rule engine; rerunning the transcripts gives both customers the same eligibility answer. section 6.1.3
Consistency controls can include:
- Changing model configuration; setting temperature to zero is given as an example. section 6.1
- Using structured output formats to reduce variation, or delegating decisions to deterministic rules. section 6.1.3 section 6.1
Figure 6.2 The three faces of unreliability in GenAI. The same ClaimAssist query produces three distinct failure modes: hallucination, where the model invents a policy clause that does not exist; robustness failure, where a minor typo flips the recommendation; and consistency failure, where two identical queries at different times return opposite answers.

Compare consistency failures with hallucination and robustness failures in the same ClaimAssist scenario. section 6.1



























