A stopping condition terminates sequential text generation once a target token appears or another limit is reached. Here, it ends decoding after the special end-of-sequence token <|endoftext|>. section 2.7
The generation loop keeps appending the next argmax token to the input sequence until that condition fires. If the eos token ID is provided, the loop breaks before yielding and decoding continues only for completed output. section 2.7
Common stopping behavior shown here includes:
- Stopping after a user-specified maximum number of new tokens. section 2.7
EOS-triggered generation
The prompt is fed into generate_text_basic_stream with eos_token_id=tokenizer.eos_token_id, so generation stops when <|endoftext|> is produced instead of running to max_new_tokens.
