Top-p filtering, also called nucleus sampling, limits which tokens an LLM considers at each generation step. It keeps the smallest set of the most probable tokens whose cumulative probability reaches the chosen threshold, then samples from that set. At top-p 0.9, lower-probability tokens outside the 90% pool are excluded. section 2.2.2
Tune top-p when a response needs a different balance of variety and focus. A higher value admits more candidate tokens; a lower value narrows the pool. This can improve coherence and reduce the risk of irrelevant or inconsistent choices, but a narrower pool does not itself establish that a response is accurate. section 2.2.2 section 2.2
Product recommendation
For a request about wireless headphones for a daily commute, top-p 0.95 may allow a detailed recommendation covering comfort, battery life, and noise cancellation. At 0.5, the response may be more focused but less comprehensive. The setting changes which wording is available to sample; it does not verify the product claims. section 2.2.2
Treat top-p as one generation control alongside temperature, maximum output length, stop sequences, and penalty strengths. Adjust it for the task’s desired response style rather than treating any single value as a guarantee against hallucinations. section 2.2 section 2.2.2
