Text categorization assigns each input message a predefined class. A GPT-like language model supports this workflow by being fine-tuned on labeled examples rather than next-token targets. section 1.7 section 6.3
Prepare the classification data as follows:
- Convert each message into token IDs with a tokenizer. section 6.3
- Truncate sequences longer than the chosen maximum and pad shorter sequences to a uniform length. section 6.3
- Pair each token sequence with a class label and load examples into training, validation, and test batches. section 6.3
Spam classification batch
A batch of eight messages can contain input tensors shaped [8, 120] and a label tensor shaped [8]. Labels are 0 for “not spam” and 1 for “spam.” section 6.3
Figure 6.7 A single training batch consisting of eight text messages represented as token IDs. Each text message consists of 120 token IDs. A class label array stores the eight class labels corresponding to the text messages, which can be either 0 (“not spam”) or 1 (“spam”).
Shows token-ID inputs alongside the corresponding class-label array in one training batch. section 6.3
