Sequence prediction
Sequence prediction is the task of predicting or generating elements of an ordered sequence from preceding elements or contextual input.
Build a Text-to-Image Generator (from Scratch)
A prediction task in which a model generates or estimates elements of an ordered sequence.
- How it is used
- Frames translation and multimodal generation as ordered, autoregressive prediction tasks.
- How deep it goes
- Explains iterative decoding, BOS and EOS tokens, attention, and multimodal pipelines, but stops short of training equations.
- Examples
- Uses German-to-English translation, image patches, and DALL-E-style text-to-image generation as concrete sequence-prediction examples.
- What you can do after
- Evaluate how models generate translations or images by tracing encoded inputs, prior outputs, next-element predictions, and stopping conditions.
Read how Build a Text-to-Image Generator (from Scratch) explains sequence prediction →
Learn Generative AI with PyTorch
A learning setup where a model predicts the next element in an ordered sequence from earlier elements.
- How it is used
- Frames text generation as predicting successive tokens from preceding context.
- How deep it goes
- Explains tokenization, indexing, shifted targets, iterative generation, and sampling, but does not derive the underlying training mathematics.
- Examples
- Uses indexed text windows, the “Anna and the” prompt, and temperature or top-K sampling.
- What you can do after
- Build and operate an LSTM text generator that predicts and appends tokens sequentially.
Read how Learn Generative AI with PyTorch explains sequence prediction →
Graph Neural Networks in Action
A prediction task in which a model uses ordered past observations to predict subsequent observations or events.
- How it is used
- Frames forecasting as predicting future node positions or events from ordered observations in dynamic graph workflows.
- How deep it goes
- Builds an RNN-based training and testing workflow, including memory, preprocessing, optimization, and multi-step rollout.
- Examples
- Uses autocomplete, search, sensor positions, pose frames, and predicted future movements as concrete examples.
- What you can do after
- Apply it via an RNN to train models and generate multi-step predictions from sensor sequences.
Read how Graph Neural Networks in Action explains sequence prediction →
Related concepts
- text-to-image modelincludes · A text-to-image model is a generative machine-learning model that produces images conditioned on textual descriptions or prompts.
- transformer architectureaddressed by · A transformer architecture is a neural-network architecture that processes sequences or other structured inputs using attention mechanisms and related transformations.
- recurrent neural networkuses · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
- next-token predictionincludes · Next-token prediction is the task or training objective of estimating the token that follows a sequence, given its preceding tokens and any conditioning information.
- sequence generationkind of · Sequence generation is the process of producing an ordered sequence of tokens, characters, or other elements, typically conditioned on input or prior elements.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.