A Keras TextVectorization layer turns strings into model-ready numbers. It can split text into words or characters, learn a vocabulary with adapt(), and produce either fixed-width multi-hot vectors or integer token sequences. Choose the output to match the model: a linear classifier can consume a multi-hot vector, while an embedding-based sequence model needs token IDs. section 14.4.1 section 15.1.1
Start with review classification
For IMDb sentiment classification, set max_tokens=20_000, split="whitespace", and output_mode="multi_hot". Adapt the layer on training reviews without labels, then map it over the training, validation, and test datasets while retaining their labels. A batch of 32 reviews becomes an array shaped (32, 20000): each row records which vocabulary terms occur in one review. That array can feed a single Dense sigmoid classifier. section 14.4.1
Listing 14.12 Applying a bag-of-words encoding to the IMDb reviews
from keras import layers max_tokens = 20_000 text_vectorization = layers.TextVectorization( max_tokens=max_tokens, split="whitespace", #1 output_mode="multi_hot", ) train_ds_no_labels = train_ds.map(lambda x, y: x) text_vectorization.adapt(train_ds_no_labels) bag_of_words_train_ds = train_ds.map( lambda x, y: (text_vectorization(x), y), num_parallel_calls=8 ) bag_of_words_val_ds = val_ds.map( lambda x, y: (text_vectorization(x), y), num_parallel_calls=8 ) bag_of_words_test_ds = test_ds.map( lambda x, y: (text_vectorization(x), y), num_parallel_calls=8 )
The listing shows the full preprocessing pattern: configure the layer, adapt it on unlabeled training text, and apply the same learned vocabulary to every dataset split. section 14.4.1
Three settings change what the layer represents:
- Add ngrams=2 to include both single words and two-word sequences in a multi-hot vocabulary. Increasing the vocabulary to 30,000 terms lets a linear IMDb classifier use phrases as well as words; its reported test accuracy rises from about 88% to 90%. section 14.4.1 section 14.4.2
- Set split="character" for character-level language modeling. With output_sequence_length=100, each 100-character chunk becomes a 100-integer sequence. Setting standardize=None preserves case and punctuation; the Shakespeare example learns a 67-character vocabulary. section 15.1.1
- Set output_mode="int" and a fixed output_sequence_length for translation inputs. Separate English and Spanish layers learn separate vocabularies; Spanish uses custom standardization so [start] and [end] remain distinct tokens while punctuation is stripped. section 15.2.1
Vectorization is also part of label preparation. In character prediction, both the input chunk and its one-character-shifted label chunk are tokenized. In English-to-Spanish translation, the Spanish token sequence is split into decoder input tokens and next-token labels; a mask excludes padded labels from loss and metrics. The layer supplies token IDs, while the dataset pipeline handles these shifts and weights. section 15.1.1 section 15.2.1
Rule of thumb: use multi-hot output when word or short-phrase presence is enough; use integer sequences when token order matters. Larger n-grams can improve a simple classifier, but long n-gram vocabularies grow unwieldy and generalize poorly. Check standardization when particular punctuation or marker tokens must survive. section 14.4.2 section 15.2.1
