A logprob scorer ranks candidate answers by their average token log probability under a language model. It scores only answer tokens, excluding the prompt, then averages rather than sums them so answers of different lengths can be compared more fairly. section 5.6
Implement the scorer by following this workflow:
- Tokenize the prompt and answer separately, concatenate their token IDs, and run the model to obtain logits. section 5.6
- Convert logits to log probabilities, select the probabilities of the answer’s next tokens, and use prompt length to locate the answer span. section 5.6
- Return the mean of the selected answer-token log probabilities as the candidate’s score. section 5.6
- Plug the scorer into self-refinement and accept a revised answer only when its score is equal to or higher than the current score. section 5.8
Comparing candidate answers
For the prompt “What is the capital of Germany?”, the answer “The capital of Germany is Berlin.” receives an average logprobability of -0.2041. The scorer can then compare it with a nonsensical alternative such as “The capital of Germany is Bridge.” In self-refinement, a revised score of -0.226 improves on -0.855, while a later score of -1.320 causes the revision to be rejected. section 5.6 section 5.8
Figure 5.18 The modified logprob scoring procedure. The prompt tokens are excluded from the calculation, and only the log probabilities of the answer tokens are collected. These values are then averaged to obtain a length-normalized score, allowing us to compare answers of different lengths fairly.
Shows how excluding prompt tokens and averaging answer-token log probabilities produces a length-normalized comparison score. section 5.6
