liveBook · Manning

Sequence prediction

Sequence prediction is the task of predicting or generating elements of an ordered sequence from preceding elements or contextual input.

How 3 Manning books explain it

Build a Text-to-Image Generator (from Scratch)

A prediction task in which a model generates or estimates elements of an ordered sequence.

How it is used
Frames translation and multimodal generation as ordered, autoregressive prediction tasks.
How deep it goes
Explains iterative decoding, BOS and EOS tokens, attention, and multimodal pipelines, but stops short of training equations.
Examples
Uses German-to-English translation, image patches, and DALL-E-style text-to-image generation as concrete sequence-prediction examples.
What you can do after
Evaluate how models generate translations or images by tracing encoded inputs, prior outputs, next-element predictions, and stopping conditions.

Read how Build a Text-to-Image Generator (from Scratch) explains sequence prediction →

Learn Generative AI with PyTorch

A learning setup where a model predicts the next element in an ordered sequence from earlier elements.

How it is used
Frames text generation as predicting successive tokens from preceding context.
How deep it goes
Explains tokenization, indexing, shifted targets, iterative generation, and sampling, but does not derive the underlying training mathematics.
Examples
Uses indexed text windows, the “Anna and the” prompt, and temperature or top-K sampling.
What you can do after
Build and operate an LSTM text generator that predicts and appends tokens sequentially.

Read how Learn Generative AI with PyTorch explains sequence prediction →

Graph Neural Networks in Action

A prediction task in which a model uses ordered past observations to predict subsequent observations or events.

How it is used
Frames forecasting as predicting future node positions or events from ordered observations in dynamic graph workflows.
How deep it goes
Builds an RNN-based training and testing workflow, including memory, preprocessing, optimization, and multi-step rollout.
Examples
Uses autocomplete, search, sensor positions, pose frames, and predicted future movements as concrete examples.
What you can do after
Apply it via an RNN to train models and generate multi-step predictions from sensor sequences.

Read how Graph Neural Networks in Action explains sequence prediction →

Also covered in

What to know first

Related concepts

Questions about sequence prediction

What is sequence prediction?

Sequence prediction is the task of predicting or generating elements of an ordered sequence from preceding elements or contextual input.

Which Manning books explain sequence prediction?

Build a Text-to-Image Generator (from Scratch), Learn Generative AI with PyTorch and Graph Neural Networks in Action.

What should I know before learning sequence prediction?

Manning books that teach sequence prediction usually introduce transformer architecture first.

What kinds of sequence prediction are there?

Kinds of sequence prediction include text-to-image model, binary sequence prediction and next-word prediction.

What is sequence prediction a kind of?

Sequence prediction is a kind of sequence generation. Sequence generation is the process of producing an ordered sequence of elements, such as tokens or characters, often conditioned on an input or context.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.