End-of-sequence token
An end-of-sequence token is a special token that marks the end of a token sequence or document and may signal that generation should stop.
Build a Text-to-Image Generator (from Scratch)
An end-of-sequence token is a special marker indicating the end of a token sequence.
- How it is used
- Marks sequence completion in tokenized inputs and tells an autoregressive translation decoder when to stop generating.
- How deep it goes
- Shows how to append and index EOS, then use its prediction as a stopping condition; does not detail how the decoder learns that prediction.
- Examples
- Uses German-to-English translation and a tokenized min-DALL-E prompt whose final ID represents </s>.
- What you can do after
- Build it from scratch by adding EOS to translation training phrases and stopping generation when the decoder predicts it.
Read how Build a Text-to-Image Generator (from Scratch) explains end-of-sequence token →
Build a Large Language Model (From Scratch)
A special token used to mark the end of a text sequence or response in GPT-style tokenization.
- How it is used
- Marks boundaries between unrelated documents and supplies padding IDs for uniform batches of variable-length messages.
- How deep it goes
- Defines the token, demonstrates vocabulary and tokenizer changes, and shows padding implementation, but does not detail model-side masking.
- Examples
- Uses concatenated sentences, token ID 1130, GPT-2 token ID 50256, tiktoken encoding, and padded spam-message batches.
- What you can do after
- Apply it via a custom tokenizer or tiktoken, insert it between documents, and configure it as a padding token in a PyTorch dataset.
Read how Build a Large Language Model (From Scratch) explains end-of-sequence token →
Learn Generative AI with PyTorch
A special end-of-sequence token used to mark completion of decoding or the end of a tokenized sequence.
- How it is used
- Uses EOS to teach a Transformer when an autoregressive translation is complete.
- How deep it goes
- Explains EOS insertion, integer mapping, and decoding termination, but does not detail its loss calculation.
- Examples
- Uses English-to-French phrases, subword pieces, vocabulary dictionaries, and generated translations as working examples.
- What you can do after
- Apply it via a Transformer decoder that stops generation when the predicted token equals EOS.
Read how Learn Generative AI with PyTorch explains end-of-sequence token →
Server-Side WebAssembly
An end-of-sequence token is a special model token indicating that generated text should terminate.
- How it is used
- Stops token-by-token story generation when the model emits its designated termination marker.
- How deep it goes
- Explains lookup, fallback behavior, bounded looping, and alternative stopping checks without detailing token-model internals.
- Examples
- Uses the prompt “Once upon a time” and a Rust loop that samples tokens until EOS or a paragraph break.
- What you can do after
- Apply it via Rust token-generation code with EOS lookup and explicit safeguards against runaway output.
Read how Server-Side WebAssembly explains end-of-sequence token →
Inside Deep Learning: Math, Algorithms, Models
A special end-of-sequence token indicating that sequence generation should stop.
- How it is used
- Marks target completion in sequence-to-sequence tasks, especially machine translation, and signals when generation can stop.
- How deep it goes
- Defines EOS and shows its placement in target sequences and vocabulary, without detailing how the model learns to emit it.
- Examples
- Appends EOS to translation targets and uses it to identify completion across variable-length outputs.
- What you can do after
- Operate sequence generation by stopping when EOS appears, or when every batch item has emitted it.
Read how Inside Deep Learning: Math, Algorithms, Models explains end-of-sequence token →
Build a Reasoning Model (From Scratch)
A special token used to mark the end of a document or generated sequence and stop generation.
- How it is used
- Stops streamed inference when the model emits the configured end-of-sequence token.
- How deep it goes
- Implements token-ID lookup, loop comparison, and early termination, without detailing token-training mechanics.
- Examples
- Uses <|endoftext|>, ID 151643, tokenizer.eos_token_id, and a one-sentence language-model prompt.
- What you can do after
- Operate/configure a streamed generator to stop cleanly at the tokenizer’s end-of-sequence ID.
Read how Build a Reasoning Model (From Scratch) explains end-of-sequence token →
Build a DeepSeek Model (From Scratch)
An end-of-text token is a special token marking the boundary between documents or text sequences.
- How it is used
- Separates pretraining documents after tokenization so concatenated streams retain explicit boundaries.
- How deep it goes
- Explains retrieval, insertion, storage, and chat-template contrast, but stops short of token-level model internals.
- Examples
- Uses TinyStories-style text samples, flattened token arrays, and the <|endoftext|> separator.
- What you can do after
- Apply it via tiktoken by appending enc.eot_token after every encoded document.
Read how Build a DeepSeek Model (From Scratch) explains end-of-sequence token →
Also covered in
- Natural Language Processing in Action, Second Edition
- How Large Language Models Work
- The AI Pocket Book
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- context windowA context window is a bounded span of sequence elements or contextual information that a model or processing operation can consider together.
- tokenA token is a discrete unit of text, such as a word, subword, punctuation mark, or symbol, produced by tokenization and processed by a language model.
- transformer architectureA transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
Related concepts
- special tokenkind of · A special token is a reserved vocabulary item that conveys structural, control, boundary, or other auxiliary information rather than ordinary content.
- tokenkind of · A token is a discrete unit of text, such as a word, subword, punctuation mark, or symbol, produced by tokenization and processed by a language model.
- generationused by · Generation is the production of an output by a computational model from supplied input or context, often using patterns learned from data.
- chat templateequivalent to · A chat template is a formatting specification that converts conversational messages into the input representation expected by a language model.
- decoder-only transformeruses · A decoder-only transformer is a Transformer language model with decoder layers and no separate encoder, generating sequence elements autoregressively from preceding context.
- decoding strategyuses · A decoding strategy is a procedure for converting an encoded or internal representation into a target output, including selecting tokens during sequence generation.
Questions about end-of-sequence token
What is end-of-sequence token?
An end-of-sequence token is a special token that marks the end of a token sequence or document and may signal that generation should stop.
Which Manning books explain end-of-sequence token?
Build a Text-to-Image Generator (from Scratch), Build a Large Language Model (From Scratch), Learn Generative AI with PyTorch, Server-Side WebAssembly, Inside Deep Learning: Math, Algorithms, Models, Build a Reasoning Model (From Scratch) and Build a DeepSeek Model (From Scratch).
What should I know before learning end-of-sequence token?
Manning books that teach end-of-sequence token usually introduce language model, context window, token and transformer architecture first.
What is end-of-sequence token a kind of?
End-of-sequence token is a kind of special token. A special token is a reserved vocabulary item that conveys structural, control, boundary, or other auxiliary information rather than ordinary content.
What is end-of-sequence token also called?
End-of-sequence token is also called EOS token, <|endoftext|> or end-of-text token.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






