Sequence model
A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.
Deep Learning with R, Third Edition
A sequence model is a model that processes an ordered sequence of tokens or other values while learning dependencies among their positions.
- How it is used
- Uses ordered token sequences for tasks such as text classification, language modeling, and machine translation.
- How deep it goes
- Explains order handling, token preprocessing, padding, truncation, architectures, and position embeddings without deriving their mathematics.
- Examples
- Demonstrates IMDb reviews, integer token sequences, fixed-length padding, RNNs, and an English-to-Spanish Transformer.
- What you can do after
- Build it from scratch by vectorizing text, preparing padded batches, and selecting an RNN, 1D convnet, or position-aware Transformer.
Read how Deep Learning with R, Third Edition explains sequence model →
Deep Learning with Python, Third Edition
A model designed to ingest an ordered token sequence and learn dependencies across positions in that sequence.
- How it is used
- Handles text classification by learning dependencies across ordered token sequences rather than discarding order.
- How deep it goes
- Defines sequence modeling, compares RNNs, 1D ConvNets, and Transformers, and details tokenization, padding, and truncation.
- Examples
- Uses IMDb reviews, integer token sequences, [PAD] tokens, and a 600-position TextVectorization pipeline.
- What you can do after
- Build it from scratch by preprocessing text into padded token-ID sequences and selecting an RNN, 1D ConvNet, or Transformer.
Read how Deep Learning with Python, Third Edition explains sequence model →
Also covered in
- Machines that ThinkIn early access (MEAP)
- Sutskever's List
What to know first
- recurrent neural networkA recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.
- convolutional neural networkA convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.
- feature spaceA feature space is a mathematical space whose dimensions represent the features used to describe data instances.
- gradient descentGradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
Related concepts
- recurrent neural networkincludes · A recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.
- n-gramincludes · An n-gram is a contiguous sequence of n consecutive items, such as words, tokens, or characters, drawn from a larger sequence.
- bag-of-words modelcontrasts with · A bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships.
- transformer architectureincludes · A transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- sequence-to-sequence modelincludes · A sequence-to-sequence model maps an input sequence to an output sequence, typically through an encoder and a decoder.
Questions about sequence model
What is sequence model?
A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.
Which Manning books explain sequence model?
Deep Learning with R, Third Edition and Deep Learning with Python, Third Edition.
What should I know before learning sequence model?
Manning books that teach sequence model usually introduce recurrent neural network, convolutional neural network, feature space and gradient descent first.
What is the difference between sequence model and bag-of-words model?
A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations. A bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships.
What kinds of sequence model are there?
Kinds of sequence model include recurrent neural network, n-gram, transformer architecture and sequence-to-sequence model.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

