A log probability expresses how likely a language model considers an event, such as a generated token. Token-level scores expose confidence signals that applications can use when grounding responses and deciding whether to trust or verify them. section 7.2.6
A practical integration can:
- Request per-token scores and alternative candidates by enabling logprobs and topLogprobs in an OpenAI chat configuration. section 7.2.6
- Inspect token, token_logprobs, and top_logprobs values to see generated-token likelihoods and competing choices. section 7.2.6
- Combine confidence metrics with a threshold and an ask-user fallback in an enhanced RAG query. section 7.2.6
Confidence-aware grounded response
A conversation handler can perform an enhanced retrieval query with a confidence threshold of 85, log returned confidence metrics, and ask the user for clarification when confidence is insufficient. section 7.2.6
"logprobs": {
"tokens": ["Regular", "exercise", "improves",
"physical", "health", ",", "boosts", "mental",
"well-being", ",", "and", "enhances", "overall",
"quality", "of", "life", "."],
"token_logprobs": [-0.5, -0.7, -0.6, -0.8,
-0.9, -1.0, -0.4, -0.5, -0.6, -0.7, -0.8, -0.9, -1.0,
-1.1, -1.2, -1.3],
"top_logprobs": [
{"token": "Regular", "logprob": -0.5},
{"token": "Consistent", "logprob": -0.6},
{"token": "Daily", "logprob": -0.7},
{"token": "exercise", "logprob": -0.7},
{"token": "activity", "logprob": -0.8},
{"token": "workout", "logprob": -0.9}
]
}
Inspect a representative response containing generated tokens, per-token log probabilities, and alternative top-log-probability candidates. section 7.2.6
