liveBook · Manning

Nucleus sampling

As explained in Applied Reinforcement Learning

Nucleus sampling draws a token from the smallest set of highest-probability tokens whose cumulative probability reaches a chosen threshold.

Top-p sampling, also called nucleus sampling, sorts candidate tokens by probability and keeps the smallest set whose probabilities sum to at least p. The next token is sampled from that set, excluding the low-probability long tail. In PPO rollouts, this stochastic choice produces varied responses for exploration while keeping them coherent. section 10.8.5

Top-p at 0.9

With top_p = 0.9, the model samples from tokens making up the top 90% of probability mass. That set might contain three tokens or fifty, depending on the model’s certainty. section 10.8.5

How Applied Reinforcement Learning covers it

How deep it goes
Defines cumulative-probability filtering and illustrates a threshold, without detailing its implementation.
Examples
Uses top_p = 0.9 and candidate sets ranging from three to fifty tokens to illustrate uncertainty-dependent selection.
What you can do after
Apply top_p = 0.9 during rollout generation to vary responses while excluding the implausible long tail.
How it connects
Connects token sampling to PPO, temperature, and policy-gradient exploration.

Also explained in

Compare how 14 books explain nucleus sampling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.