Sequence model
A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.
Deep Learning with R, Third Edition
A sequence model is a model that processes an ordered sequence of tokens or other values while learning dependencies among their positions.
- How it is used
- Uses ordered token sequences for tasks such as text classification, language modeling, and machine translation.
- How deep it goes
- Explains order handling, token preprocessing, padding, truncation, architectures, and position embeddings without deriving their mathematics.
- Examples
- Demonstrates IMDb reviews, integer token sequences, fixed-length padding, RNNs, and an English-to-Spanish Transformer.
- What you can do after
- Build it from scratch by vectorizing text, preparing padded batches, and selecting an RNN, 1D convnet, or position-aware Transformer.
Read how Deep Learning with R, Third Edition explains sequence model →
Deep Learning with Python, Third Edition
A model designed to ingest an ordered token sequence and learn dependencies across positions in that sequence.
- How it is used
- Handles text classification by learning dependencies across ordered token sequences rather than discarding order.
- How deep it goes
- Defines sequence modeling, compares RNNs, 1D ConvNets, and Transformers, and details tokenization, padding, and truncation.
- Examples
- Uses IMDb reviews, integer token sequences, [PAD] tokens, and a 600-position TextVectorization pipeline.
- What you can do after
- Build it from scratch by preprocessing text into padded token-ID sequences and selecting an RNN, 1D ConvNet, or Transformer.
Read how Deep Learning with Python, Third Edition explains sequence model →
Related concepts
- recurrent neural networkincludes · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
- n-gramincludes · An n-gram is a contiguous sequence of n consecutive items, commonly linguistic tokens or characters, drawn from a larger sequence.
- bag-of-words modelcontrasts with · A bag-of-words model represents text using counts, presence indicators, or weights for its words or tokens while disregarding their order and syntactic relationships.
- transformer architectureincludes · A transformer architecture is a neural-network architecture that processes sequences or other structured inputs using attention mechanisms and related transformations.
- pretrainingaddressed by · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.