Log probabilities are completion metadata containing the natural logarithms of the probabilities assigned to candidate tokens. For each generated token, the API can expose the selected token’s value and the most likely alternatives. section 3.3.4
The completion API returns log probabilities as token-aligned arrays. Set logprobs to an integer from 0 through 5 to request alternatives; the sampled token is always included, so an array can contain one additional entry. section 3.2.1 section 3.3.4
Use log probabilities to:
- Inspect which alternative tokens the model considered when a completion is unsatisfactory. section 3.3.4
- Assess uncertainty: similar probabilities for different tokens indicate that the next token is not clearly determined. section 3.3.4
- Support selection among multiple completions; best_of chooses the completion with the highest log probability per token. section 3.2.3
Inspecting a dog-name completion
A request for a one-word name for a white miniature poodle can return Cotton. With logprobs=3, the response exposes token-level values and alternatives such as Casper, Coco, and Snow, making the generation easier to inspect. section 3.3.4
Listing 3.18 Output log probabilities
{
"id": "cmpl-7giPQGlKc6c7BaWmHgOLyZqabIruw",
"object": "text_completion",
"created": 1690414840,
"model": "gpt-35-turbo",
"choices": [
{
"text": "\n\nCotton",
"index": 0,
"finish_reason": "stop",
"logprobs": {
"tokens": [
"\n",
"\n",
"C",
"otton"
],
"token_logprobs": [
-0.0008873215,
-4.361666e-06,
-1.026479,
-0.56846446
],
"top_logprobs": [
{
"\n": -0.0008873215,
"\n\n": -7.660001,
" Angel": -10.180796
},
{
"\n": -4.361666e-06,
"\n\n": -12.970553,
"<|endoftext|>": -15.136529
},
{
"C": -1.026479,
"P": -2.255978,
"Snow": -2.1068947
},
{
"asper": -2.001854,
"oco": -1.957575,
"otton": -0.56846446
}
],
"text_offset": [
54,
55,
56,
57
]
}
}
],
"usage": {
"completion_tokens": 4,
"prompt_tokens": 12,
"total_tokens": 16
}
}
Inspect the token sequence, selected-token log probabilities, and top alternative tokens in a completion response. section 3.3.4
Request log probabilities judiciously: they increase generated response data, API cost, and latency. Combine them with temperature when balancing creativity, determinism, and prompt quality. section 3.3.4 section 6.8
