Log probability grounding estimates response reliability by examining the probability assigned to each generated token. A first-token score can indicate whether the model believes it has enough information to answer. section 7.2.6
A practical implementation exposes token-level confidence data:
- Enable log probabilities and request alternative token probabilities through the model configuration. section 7.2.6
- Inspect generated tokens, token log probabilities, and competing top-token probabilities. section 7.2.6
- Use confidence metrics and a threshold to decide whether to answer or ask the user for clarification. section 7.2.6
Confidence-aware RAG response
A RAG helper performs an enhanced query with an 85 confidence threshold and an ask-user fallback, then logs returned confidence metrics before continuing the conversation. section 7.2.6
"logprobs": {
"tokens": ["Regular", "exercise", "improves",
"physical", "health", ",", "boosts", "mental",
"well-being", ",", "and", "enhances", "overall",
"quality", "of", "life", "."],
"token_logprobs": [-0.5, -0.7, -0.6, -0.8,
-0.9, -1.0, -0.4, -0.5, -0.6, -0.7, -0.8, -0.9, -1.0,
-1.1, -1.2, -1.3],
"top_logprobs": [
{"token": "Regular", "logprob": -0.5},
{"token": "Consistent", "logprob": -0.6},
{"token": "Daily", "logprob": -0.7},
{"token": "exercise", "logprob": -0.7},
{"token": "activity", "logprob": -0.8},
{"token": "workout", "logprob": -0.9}
]
}
Inspect a representative response containing generated tokens, token_logprobs, and alternative top_logprobs. section 7.2.6
