A hallucination is a false or unsupported output from a large language model (LLM) that is stated clearly enough to appear authoritative. Fluency and confidence do not establish truth: an LLM can describe something convincingly even when it does not exist. Treat every output as predictive and subject to verification. section 2.2.1 section 2.8
Hallucinations arise or become more likely when:
- Training data contains errors, gaps, or assumptions, or the model is overfitted. section 2.2.1 section 2.8
- The model prioritizes giving an answer instead of acknowledging that it cannot provide useful information. section 2.2.1 section 4.4.5
- A conversation becomes longer and less precise, encouraging the model to overfit its responses to increasingly broad requests. section 2.7
- Repeated requests for additional ideas exhaust relevant suggestions and produce less connected results. section 4.2.2 section 4.4.5
Reduce risk by defining a specific task, supplying relevant context, stating the required format, using examples or delimiters, checking assumptions, and splitting complex work into subtasks. Prompt quality improves the odds of a useful answer; it does not remove the need for critical judgment. section 2.8 section 10.1
A convincing but false recommendation
A request for an introduction produced a polished recommendation for a supposedly relevant testing title. The title did not exist, but the wording made the claim sound reliable. The appropriate response is skepticism and independent verification, not acceptance based on fluency. section 2.2.1
A false code-analysis risk
An LLM claimed that submitTimesheet failed to handle zero or negative durations. The method actually rejected those values and returned false. The example shows why generated risks must be checked against the code before being treated as defects. section 4.4.5
###
public class Timesheet {
private HashMap<String, Integer> timesheets = new HashMap<>();
public boolean submitTimesheet(String projectName, int duration) {
if (projectName != null && duration > 0) {
// Check to see if project already exists
if (timesheets.containsKey(projectName)) {
// If it does, add the new duration to the
// existing total
duration += timesheets.get(projectName);
}
timesheets.put(projectName, duration);
return true;
} else {
return false;
}
}
public Collection<Integer> getTimesheets() {
return timesheets.values();
}
public int getTotalTimesheetHours(String projectName) {
return timesheets.get(projectName);
}
}
###
Inspect the timesheet implementation directly to verify whether the generated validation claim matches the executed condition. section 4.4.5
