A fluent answer can still contain a wrong price, an invented policy, or an unsupported product claim. An LLM-as-a-judge evaluator uses a second model to assess a generated response against specified criteria. It can identify nuanced errors that token-overlap metrics such as ROUGE may miss, but its verdict is evidence for review—not proof of correctness. section 9.1.4 section 9.0
Checking a roaming-fee answer
A customer asks about international roaming fees. Supply the evaluator with the question, retrieved policy documents, and the chatbot’s answer. Ask whether the per-megabyte rates match the official fee schedule, whether the answer addresses the question, and whether any claims lack support. A low factual-correctness score paired with a specific rate discrepancy flags the response for investigation. Running the check across many responses can reveal recurring problems to fix in the RAG system. section 3.6.1
A practical judging workflow separates the evidence, rubric, and decision:
- Provide the generated response and a reference answer or retrieved documents. Ask the judge to identify claims unsupported or contradicted by that evidence. section 9.1.4 section 3.6.1
- Request an actionable result: a hallucination flag, severity score, explanation, and list of disputed claims. For RAG, factual correctness, contextual appropriateness, and grounding can also receive separate scores. section 9.1.4 section 3.6.1
- Review suspicious results rather than treating scores as ground truth. Judges can hallucinate, favor longer or stylistically familiar answers, and overlook subtle factual errors. Multiple judges with majority voting, human checks on a random subset, and retrieval-based fact-checking against structured databases provide complementary checks. section 9.1.4
from openai import OpenAI
from pydantic import BaseModel
client = OpenAI()
class HallucinationJudgment(BaseModel):
has_hallucination: bool
severity: int # 1-5 scale
explanation: str
hallucinated_claims: list[str]
def llm_judge_hallucination(reference: str, generated: str):
prompt = f'''Evaluate the generated text for hallucinations.
Reference (ground truth): {reference}
Generated response: {generated}
Identify claims not supported or contradicted by reference.'''
response = client.beta.chat.completions.parse(
model='gpt-5.4',
messages=[{'role': 'user', 'content': prompt}],
response_format=HallucinationJudgment
)
return response.choices[0].message.parsed
The Python listing prompts an OpenAI model with a reference and generated response, then parses the verdict into a Pydantic schema. Its boolean, severity, explanation, and claim list make individual decisions inspectable; the schema does not establish that the decisions are correct. section 9.1.4
Judging can serve different points in a system. Post-hoc evaluation helps monitor answer quality alongside reference metrics and human review. Semantic guardrails can judge whether agent content is safe and on topic where keyword filters are brittle. During constitutional AI training, a critic evaluates drafts against rules and triggers rewrites; the judgment changes the training process rather than merely flagging a deployed answer. section 1.4.3 section 6.4.2 section 11.3.3
