Hallucination is a language-model failure mode in which generated text sounds plausible but is unsupported or factually wrong. A model always predicts some next token, even when the prompt has no grounded answer. section 16.3.1 section 16.6
Transformers store information in a continuous vector space rather than discrete database entries. Interpolation can support useful generalization, but it can also produce inaccurate or meaningless combinations. section 15.5
Practical implications include:
- Treat generated claims as potentially unreliable, because all LLMs occasionally produce factually incorrect information. section 16.6
- Use instruction fine-tuning with instruction-and-response examples to make outputs better suited to a target conversation or task. section 16.3.1 section 16.6
- Recognize that prompting and sampling can explore a model’s distribution but do not provide precise control or a guaranteed cure. section 16.3.1 section 16.6
A confident false answer
When prompted, “Tell me about the 542nd president of the United States,” Gemma replies, “The 542nd president of the United States was James A. Garfield.” The fluent completion illustrates how a model can continue an impossible premise instead of identifying it as unsupported. section 16.3.1
gemma_lm$generate( "Tell me about the 542nd president of the United States.", max_length = 40L ) |> cat() Tell me about the 542nd president of the United States. The 542nd president of the United States was James A. Garfield.
Inspect the complete impossible-president prompt and the model’s factually incorrect completion. section 16.3.1
