A hallucination is an LLM response that presents incorrect, fabricated, nonsensical, or source-unsupported content with the confidence of an accurate answer. Unlike a crash or error message, it can look useful enough to trust. Forms include invented facts or citations, claims that contradict supplied context, and claims that contradict world knowledge. section 1.3.2 section 9.1
LLMs generate likely next tokens, not verified facts. They can reproduce the style of authoritative sources without checking whether a particular claim is true. Gaps, errors, and contradictions in training data make plausible fabrication more likely when the answer is unknown. section 1.3.2
An answer that looks right
A customer asks about a product warranty. The reference says two years; the response confidently says three. That mismatch is a hallucination even if the sentence is fluent and otherwise helpful. An invented court citation poses the same problem in a higher-stakes setting. section 9.1.6 section 9.1
Reduce risk at the point where an answer is produced:
- Provide clear context, instructions, and examples in the prompt. Tune settings such as temperature and top-p, but do not treat predictable wording as proof of correctness. section 2.0
- Ground claims in retrieved evidence with retrieval-augmented generation (RAG), rather than relying only on model memory. section 3.3
- Use an external API for information that requires a current answer. A weather agent that checks an API has evidence a model guessing from training data lacks. section 6.2.3
- Limit response length and conversation turns when they encourage off-topic generation. Test the limits: overly short responses can omit needed detail. section 2.3
Measure hallucinations against references before deployment and on a recurring sample of production outputs. An LLM judge can identify unsupported claims and explain its verdict, but judges can also miss errors or mislabel correct answers. Combine automated judgments with human checks or retrieval-based fact-checking, then track the hallucination rate and alert on threshold breaches. section 9.1.4 section 9.1.6
from phoenix.evals import (
HALLUCINATION_PROMPT_TEMPLATE,
HALLUCINATION_PROMPT_RAILS_MAP,
OpenAIModel, llm_classify
)
import pandas as pd
df = pd.DataFrame({
'reference': ['Product has 2-year warranty...'],
'response': ['The product comes with a 3-year warranty...']
})
model = OpenAIModel(model_name='gpt-5.4')
rails = list(HALLUCINATION_PROMPT_RAILS_MAP.values())
results = llm_classify(
dataframe=df,
template=HALLUCINATION_PROMPT_TEMPLATE,
model=model, rails=rails,
provide_explanation=True
)
A Phoenix evaluation listing compares reference and response columns, returning a structured label and explanation. Its two-year versus three-year warranty pair makes the unsupported claim inspectable. section 9.1.6
