Model self-awareness is an LLM’s ability to recognize when it cannot solve a problem because it lacks needed information or capability. Recognizing that limit can help an agent choose a tool instead of guessing. section 2.4.3
Common limits include:
- Missing current or verifiable information, such as a value that must be checked on a web page. section 2.4.3
- Missing access to attached files, such as a spreadsheet or document. section 2.4.3
Kipchoge problem
A question requiring the minimum perigee value from the Moon’s Wikipedia page calls for web access; recalling a value from training data is not a substitute for checking it. section 2.4.3
Table 2.1 Experiment results across models
| Model |
Accuracy |
Judged solvable |
|---|---|---|
| GPT-5 |
10/20 (50%) |
11/20 (55%) |
| GPT-5 mini |
4/20 (20%) |
8/20 (40%) |
| Claude Sonnet 4.5 |
3/20 (15%) |
4/20 (20%) |
| Claude Haiku 4.5 |
3/20 (15%) |
4/20 (20%) |
Compare model accuracy with judgments about whether problems were solvable. section 2.4.3
