liveBook · Manning

Sequence model

A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.

How 2 Manning books explain it

Deep Learning with R, Third Edition

A sequence model is a model that processes an ordered sequence of tokens or other values while learning dependencies among their positions.

How it is used
Uses ordered token sequences for tasks such as text classification, language modeling, and machine translation.
How deep it goes
Explains order handling, token preprocessing, padding, truncation, architectures, and position embeddings without deriving their mathematics.
Examples
Demonstrates IMDb reviews, integer token sequences, fixed-length padding, RNNs, and an English-to-Spanish Transformer.
What you can do after
Build it from scratch by vectorizing text, preparing padded batches, and selecting an RNN, 1D convnet, or position-aware Transformer.

Read how Deep Learning with R, Third Edition explains sequence model →

Deep Learning with Python, Third Edition

A model designed to ingest an ordered token sequence and learn dependencies across positions in that sequence.

How it is used
Handles text classification by learning dependencies across ordered token sequences rather than discarding order.
How deep it goes
Defines sequence modeling, compares RNNs, 1D ConvNets, and Transformers, and details tokenization, padding, and truncation.
Examples
Uses IMDb reviews, integer token sequences, [PAD] tokens, and a 600-position TextVectorization pipeline.
What you can do after
Build it from scratch by preprocessing text into padded token-ID sequences and selecting an RNN, 1D ConvNet, or Transformer.

Read how Deep Learning with Python, Third Edition explains sequence model →

Also covered in

What to know first

Related concepts

Questions about sequence model

What is sequence model?

A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.

Which Manning books explain sequence model?

Deep Learning with R, Third Edition and Deep Learning with Python, Third Edition.

What should I know before learning sequence model?

Manning books that teach sequence model usually introduce recurrent neural network, convolutional neural network, feature space and gradient descent first.

What is the difference between sequence model and bag-of-words model?

A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations. A bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships.

What kinds of sequence model are there?

Kinds of sequence model include recurrent neural network, n-gram, transformer architecture and sequence-to-sequence model.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.