A classifier assigns an input to one of a fixed set of categories. A binary classifier chooses between two labels; a multicategory classifier chooses among more than two. For text generation, predicting the next token is a multicategory classification task: the possible labels are tokens in the vocabulary. This is different from assigning one category to an entire text sequence. section 2.5 section 12.1.2
Start with two clothing labels
To build a small binary classifier, keep only t-shirts and ankle boots from Fashion MNIST. Convert their images to tensors and load shuffled batches of 64. A dense PyTorch network flattens each 28 × 28 image to 784 values, passes it through three ReLU hidden layers, and produces one sigmoid output: the predicted probability of an ankle boot. Train it with binary cross-entropy and Adam. At test time, a 0.5 cutoff turns that probability into a label; compare labels with the test set to calculate accuracy. The reported test accuracy is 87.84%. section 2.3.1 section 2.3.2 section 2.3.3
Listing 2.5 Creating a binary classification model
import torch.nn as nn
device="cuda" if torch.cuda.is_available() else "cpu" #1
binary_model=nn.Sequential( #2
nn.Linear(28*28,256), #3
nn.ReLU(), #4
nn.Linear(256,128),
nn.ReLU(),
nn.Linear(128,32),
nn.ReLU(),
nn.Linear(32,1),
nn.Dropout(p=0.25),
nn.Sigmoid()).to(device) #5
The PyTorch listing shows how dense layers, ReLU activations, and a single sigmoid output form the binary classifier. section 2.3.2
For ten clothing categories, retain all Fashion MNIST labels instead of filtering down to two. Cross-entropy loss compares predicted category probabilities with the true labels. Split training data into training and validation sets so validation loss can guide when to stop: continuing after it stops improving risks overfitting. section 2.2.1 section 2.4.1
The same classification view helps explain generation. An LSTM, a text Transformer, or a music Transformer predicts the next token from its vocabulary, compares that prediction with the actual next token using cross-entropy loss, and updates its parameters. Repeating token prediction can then produce a sequence. This is next-token classification, not a worked example of categorizing complete documents. section 8.1.3 section 12.1.2 section 14.1.3
