liveBook · Manning

Bag-of-words model

Also called BOW, bag-of-words representation

A bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships.

How 6 Manning books explain it

Natural Language Processing in Action, Second Edition

Bag-of-words is a text representation that records word occurrences while disregarding the order and syntactic relationships of the words.

How it is used
Represents documents as numerical vectors for search, retrieval, sentiment analysis, and other NLP tasks.
How deep it goes
Works through tokenization, vocabulary, binary and count vectors, and relative frequencies, while highlighting lost word order.
Examples
Counts spaCy tokens in an algorithmic-bias sentence and compares permutations of a greeting with a logical query.
What you can do after
Apply it via Python’s Counter to inspect token counts and calculate relative frequencies.

Read how Natural Language Processing in Action, Second Edition explains bag-of-words model →

Deep Learning with Python, Third Edition

A text model that ignores token order and represents a document by which words occur in it, often with per-word weights.

How it is used
Supports lightweight text classification by converting documents into unordered vocabulary features.
How deep it goes
Defines token sets, multi-hot vectors, learned weights, and a linear classifier, but stops short of modeling word order.
Examples
Uses a short movie sentence and IMDb sentiment reviews, including a 20,000-term vector and an approximately 88% test accuracy.
What you can do after
Build it from scratch with TextVectorization and a linear classifier for lightweight sentiment prediction.

Read how Deep Learning with Python, Third Edition explains bag-of-words model →

Data Without Labels

Bag-of-words is a text representation that uses word occurrences or frequencies as features while disregarding word order and context.

How it is used
Represents text quickly as frequency vectors for document classification and other text-analysis tasks.
How deep it goes
Explains tokenization, vocabulary creation, frequency vectors, and thresholds, but stops short of implementation code and contextual modeling.
Examples
Uses two short sentences to demonstrate token frequencies, vocabulary-sized vectors, and zero values for absent words.
What you can do after
Apply the model to convert sentences or documents into frequency vectors for fast document classification.

Read how Data Without Labels explains bag-of-words model →

Transfer Learning for Natural Language Processing

A bag-of-words model represents text by its words and their counts or weights while disregarding word order.

How it is used
Turns email text into numerical features for a spam-classification baseline.
How deep it goes
Works through vocabulary construction and count-table generation, then notes sparsity and briefly compares n-grams and tf-idf.
Examples
Uses word-count features from a 2,000-email spam dataset.
What you can do after
Build it from scratch by counting token occurrences into email-by-vocabulary features.

Read how Transfer Learning for Natural Language Processing explains bag-of-words model →

Machine Learning System Design

A bag-of-words representation describes text using counts or indicators of its words while disregarding word order.

How it is used
Transforms sequential data such as text into features that machine-learning algorithms can process.
How deep it goes
Defines the representation and its word-order limitation, then points to longer N-grams and alternatives rather than implementation details.
Examples
Uses text as the input sequence and contrasts single-word unigrams with longer N-grams.
What you can do after
Apply it to transform text into machine-learning features while recognizing the loss of word-order information.

Read how Machine Learning System Design explains bag-of-words model →

Deep Learning with R, Third Edition

A bag-of-words model represents text as an unordered collection of tokens and assigns learned weights to token presence indicators.

How it is used
Deploys unordered token indicators as a simple baseline for movie-review sentiment classification.
How deep it goes
Explains tokenization, set conversion, multi-hot encoding, and learned word weights, but stops short of detailed model architecture.
Examples
Uses a toy review about crying, vocabulary IDs, an unordered token set, and a six-position multi-hot vector.
What you can do after
Apply it via tokenization and multi-hot encoding to create fixed-size inputs for a text-classification model.

Read how Deep Learning with R, Third Edition explains bag-of-words model →

Also covered in

What to know first

Related concepts

Questions about bag-of-words model

What is bag-of-words model?

A bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships.

Which Manning books explain bag-of-words model?

Natural Language Processing in Action, Second Edition, Deep Learning with Python, Third Edition, Data Without Labels, Transfer Learning for Natural Language Processing, Machine Learning System Design and Deep Learning with R, Third Edition.

What should I know before learning bag-of-words model?

Manning books that teach bag-of-words model usually introduce n-gram, term frequency-inverse document frequency, embedding and Word2Vec first.

What is the difference between bag-of-words model and sequence model?

A bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships. A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.

What is bag-of-words model also called?

Bag-of-words model is also called BOW or bag-of-words representation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.