Top-p filtering, also called nucleus sampling, generates varied text by sampling only from the smallest group of token candidates whose cumulative probability reaches a chosen threshold. Unlike top-k sampling, the group size changes with the probability distribution. section 10.2.1
Sampling GPT-2 text
With do_sample=True, max_length=50, and top_p=0.92, a GPT-2 prompt such as “NLP is a” produces probabilistic output about network protocols. Repeated runs can differ, and top-p filtering does not make the text domain-specific; fine-tuning remains necessary. section 10.2.1
Listing 10.7 Generating text using the nucleus sampling method
>>> nucleus_sampling_args = {
... 'do_sample': True,
... 'max_length': 50,
... 'top_p': 0.92
... }
>>> print(generate(prompt='NLP is a', **nucleus_sampling_args))
NLP is a multi-level network protocol, which is one of the most
well-documented protocols for managing data transfer protocols. This
is useful if one can perform network transfers using one data transfer
protocol and another protocol or protocol in the same chain.
Inspect the runnable Hugging Face generation arguments and resulting nucleus-sampled output. section 10.2.1
