A medical answer can sound confident and still be wrong. Uncertainty is the lack of confidence or knowledge about whether an answer is correct. In radiology board-exam questions, GPT-4 used similarly confident language for correct and incorrect answers, making its mistakes hard to spot from its wording alone. section 5.8
This matters especially when a novice practitioner might trust an answer without checking it. Even a model that is correct 99% of the time can be dangerous if it hallucinates in the remaining cases and gives no sign that it may be wrong. Modeling uncertainty means helping users recognize those limits, not just improving accuracy. section 5.8
Confidence is not the only concern: a medical-evidence summary can be accurate yet misleading if it leaves out the most important points. Human clinicians can also be overconfident. Both the answer and its apparent certainty need scrutiny. section 5.8
