liveBook · Manning

End-of-sequence token

Also called EOS token, <|endoftext|>, end-of-text token

An end-of-sequence token is a special token that marks the end of a token sequence or document and may signal that generation should stop.

How 7 Manning books explain it

Build a Text-to-Image Generator (from Scratch)

An end-of-sequence token is a special marker indicating the end of a token sequence.

How it is used
Marks sequence completion in tokenized inputs and tells an autoregressive translation decoder when to stop generating.
How deep it goes
Shows how to append and index EOS, then use its prediction as a stopping condition; does not detail how the decoder learns that prediction.
Examples
Uses German-to-English translation and a tokenized min-DALL-E prompt whose final ID represents </s>.
What you can do after
Build it from scratch by adding EOS to translation training phrases and stopping generation when the decoder predicts it.

Read how Build a Text-to-Image Generator (from Scratch) explains end-of-sequence token →

Build a Large Language Model (From Scratch)

A special token used to mark the end of a text sequence or response in GPT-style tokenization.

How it is used
Marks boundaries between unrelated documents and supplies padding IDs for uniform batches of variable-length messages.
How deep it goes
Defines the token, demonstrates vocabulary and tokenizer changes, and shows padding implementation, but does not detail model-side masking.
Examples
Uses concatenated sentences, token ID 1130, GPT-2 token ID 50256, tiktoken encoding, and padded spam-message batches.
What you can do after
Apply it via a custom tokenizer or tiktoken, insert it between documents, and configure it as a padding token in a PyTorch dataset.

Read how Build a Large Language Model (From Scratch) explains end-of-sequence token →

Learn Generative AI with PyTorch

A special end-of-sequence token used to mark completion of decoding or the end of a tokenized sequence.

How it is used
Uses EOS to teach a Transformer when an autoregressive translation is complete.
How deep it goes
Explains EOS insertion, integer mapping, and decoding termination, but does not detail its loss calculation.
Examples
Uses English-to-French phrases, subword pieces, vocabulary dictionaries, and generated translations as working examples.
What you can do after
Apply it via a Transformer decoder that stops generation when the predicted token equals EOS.

Read how Learn Generative AI with PyTorch explains end-of-sequence token →

Server-Side WebAssembly

An end-of-sequence token is a special model token indicating that generated text should terminate.

How it is used
Stops token-by-token story generation when the model emits its designated termination marker.
How deep it goes
Explains lookup, fallback behavior, bounded looping, and alternative stopping checks without detailing token-model internals.
Examples
Uses the prompt “Once upon a time” and a Rust loop that samples tokens until EOS or a paragraph break.
What you can do after
Apply it via Rust token-generation code with EOS lookup and explicit safeguards against runaway output.

Read how Server-Side WebAssembly explains end-of-sequence token →

Inside Deep Learning: Math, Algorithms, Models

A special end-of-sequence token indicating that sequence generation should stop.

How it is used
Marks target completion in sequence-to-sequence tasks, especially machine translation, and signals when generation can stop.
How deep it goes
Defines EOS and shows its placement in target sequences and vocabulary, without detailing how the model learns to emit it.
Examples
Appends EOS to translation targets and uses it to identify completion across variable-length outputs.
What you can do after
Operate sequence generation by stopping when EOS appears, or when every batch item has emitted it.

Read how Inside Deep Learning: Math, Algorithms, Models explains end-of-sequence token →

Build a Reasoning Model (From Scratch)

A special token used to mark the end of a document or generated sequence and stop generation.

How it is used
Stops streamed inference when the model emits the configured end-of-sequence token.
How deep it goes
Implements token-ID lookup, loop comparison, and early termination, without detailing token-training mechanics.
Examples
Uses <|endoftext|>, ID 151643, tokenizer.eos_token_id, and a one-sentence language-model prompt.
What you can do after
Operate/configure a streamed generator to stop cleanly at the tokenizer’s end-of-sequence ID.

Read how Build a Reasoning Model (From Scratch) explains end-of-sequence token →

Build a DeepSeek Model (From Scratch)

An end-of-text token is a special token marking the boundary between documents or text sequences.

How it is used
Separates pretraining documents after tokenization so concatenated streams retain explicit boundaries.
How deep it goes
Explains retrieval, insertion, storage, and chat-template contrast, but stops short of token-level model internals.
Examples
Uses TinyStories-style text samples, flattened token arrays, and the <|endoftext|> separator.
What you can do after
Apply it via tiktoken by appending enc.eot_token after every encoded document.

Read how Build a DeepSeek Model (From Scratch) explains end-of-sequence token →

Also covered in

What to know first

Related concepts

Questions about end-of-sequence token

What is end-of-sequence token?

An end-of-sequence token is a special token that marks the end of a token sequence or document and may signal that generation should stop.

Which Manning books explain end-of-sequence token?

Build a Text-to-Image Generator (from Scratch), Build a Large Language Model (From Scratch), Learn Generative AI with PyTorch, Server-Side WebAssembly, Inside Deep Learning: Math, Algorithms, Models, Build a Reasoning Model (From Scratch) and Build a DeepSeek Model (From Scratch).

What should I know before learning end-of-sequence token?

Manning books that teach end-of-sequence token usually introduce language model, context window, token and transformer architecture first.

What is end-of-sequence token a kind of?

End-of-sequence token is a kind of special token. A special token is a reserved vocabulary item that conveys structural, control, boundary, or other auxiliary information rather than ordinary content.

What is end-of-sequence token also called?

End-of-sequence token is also called EOS token, <|endoftext|> or end-of-text token.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.