A hallucination is an LLM output that sounds right but is wrong or made up. In a production application, that is more than an odd answer: a plausible citation or link can mislead a user who has no reason to question it. Confident wording is not evidence of accuracy. section 3.2.7
Plausible books, nonexistent links
Asked for books from a particular publisher, an LLM supplied one nonexistent title and one real title from a different publisher. Both links looked like publisher URLs but led to 404 errors. The titles, authors, and URL shapes made the answers seem credible without making them correct. section 3.2.7
There is no established single cause or complete fix. Models can sound certain even when mistaken, and inconsistent answers are a substantial obstacle for applications that need reliable results. Treat factual claims as claims to check, especially when users will act on them. section 3.2.7 section 12.2.5
Ways to improve reliability include:
- Provide relevant context through prompting; it can make responses more consistent, but does not make them inherently trustworthy. section 1.3
- Evaluate prompt changes against answers you can check. A DSPy math example uses Chain of Thought and bootstrapped few-shot prompts without changing tokenization or fine-tuning the model. section 12.2.5
- Check what leaves the application as well as what enters it: outputs can be outdated, factually incorrect, or contain copyrighted or licensed material. section 1.3
config = dict(max_bootstrapped_demos=2) #1
class CoT(dspy.Module):
def __init__(self):
super().__init__()
self.prog = dspy.ChainOfThought(QASignature, max_tokens=1000)
def forward(self, question):
return self.prog(question=question)
print("Creating Bootstrapped Few Shot Prompt") #2
teleprompter = BootstrapFewShot(metric=gsm8k_metric, **config)
optimized_cot = teleprompter.compile(
CoT(), trainset=gsm8k_trainset, valset=gsm8k_devset
)
optimized_cot.save("optimized_llama3_math_cot.json")
print("Evaluating Optimized CoT Prompt") #3
evaluate(optimized_cot)
#149/200 74.5%
The listing shows how a bootstrapped few-shot Chain of Thought prompt is compiled and evaluated for math questions; it is an example of testing a mitigation, not a general cure for hallucinations. section 12.2.5
