An open-domain task answers without a specific grounding document or supplied source. The model relies on parametric memory, external search, or another knowledge base, so evaluation centers on factual alignment rather than contextual grounding. section 5.1 section 5.1.1
Evaluate open-domain outputs by:
- Checking consistency across independent generations, while recognizing that consistently learned errors can escape detection. section 5.5
- Retrieving external facts or decomposing responses into atomic claims for independent verification. section 5.1 section 5.5
- Treating factual-alignment failure as a quality gate before grading other response dimensions. section 5.5
A factual question without a source
A chatbot answering “Who founded OpenAI?” draws on knowledge acquired during pre-training rather than a supplied document. Verification therefore requires factual-alignment signals or external evidence. section 5.1.1
Figure 5.1 A flow chart for choosing a hallucination detection strategy. Closed-domain tasks can use grounding based on sources to verify claims. Open-domain tasks require self-consistency checks, or rely on external fact retrieval, or other factual alignment signals.
Use the decision flow to distinguish open-domain tasks from source-grounded tasks and select consistency checks, external retrieval, or other factual-alignment signals. section 5.1

