Self-consistency improves inference accuracy by generating several candidate answers for one prompt and choosing the most frequent final answer. It changes no model weights, but spends more computation at answer-generation time. section 4.1 section 4.6
Implement the voting pipeline in three steps:
- Generate multiple responses with temperature greater than zero and top-p filtering so each sample can differ. section 4.4
- Extract a comparable final answer, such as the boxed result, from every response. section 4.6
- Run samples sequentially with a loop or parallelize them across devices when latency matters; either way, compute and cost increase. section 4.6 section 4.7
Five-sample arithmetic vote
For one equation, five generated responses produce 83, 22, 54, 83, and 61. Frequency voting returns 83, the only repeated candidate. A typical call uses num_samples=5, temperature=0.8, top_p=0.9, max_new_tokens=2048, and an optional per-round seed.
Diversity comes from multinomial next-token sampling, with temperature reshaping probabilities and top-p filtering removing low-probability tokens before sampling. Chain-of-thought prompting can be combined with the vote, often improving accuracy while increasing output length and runtime. section 4.4.3 section 4.7
