A bag-of-words model represents text as an unordered collection of tokens. It discards word order and learns a weight for each vocabulary term, making it a simple alternative to sequence models. section 14.3 section 14.4
From review to vector
For “this movie made me cry,” tokenization produces token IDs; removing order yields a set such as {0, 1, 3, 5}; multi-hot encoding then marks each vocabulary word as present or absent. The resulting fixed-size vector can train a classifier directly. section 14.4
A practical implementation follows this path:
- Use Keras TextVectorization to tokenize text, learn a vocabulary with adapt(), and emit multi-hot vectors. section 14.4.1
- Feed the vectors to a linear Dense classifier with one learned parameter per vocabulary term plus a bias. section 14.4.1
- Use early stopping when validation loss stops improving; the compact model trains quickly but cannot capture word-order effects. section 14.4.1 section 14.3
"this movie made me cry" {"[UNK]": 0, "movie": 1, "film": 2, "made": 3, "laugh": 4, "cry": 5}
Use the vocabulary listing to trace how a short review becomes token IDs before order is discarded and presence is encoded. section 14.4
