A classifier assigns an input to one or more predefined categories. In image classification, a vision transformer (ViT) converts an image into tokens, processes them with self-attention, and predicts an image class. section 3.0 section 3.1
A ViT classifier follows a concrete sequence:
- Split each 32 × 32 CIFAR-10 image into an 8 × 8 grid, producing 64 image patches. section 3.0
- Encode patches as tokens and add spatial information so the transformer can process visual structure. section 3.0 section 3.1
- Pass the token sequence through an encoder-only transformer whose self-attention captures local and global image features. section 3.0
- Train the model to distinguish categories such as airplanes, cars, birds, and cats. section 3.0
From general classifier to detector
A pretrained ResNet50 can become a real-versus-fake image classifier by freezing early layers, replacing its output head, training the new layers on labeled images, and monitoring validation performance with early stopping. section 13.4
Figure 3.1 The eight-step road map for building a text-to-image generator from scratch. In this chapter, you’ll focus on step 2: building a ViT for image classification. This is your bridge from transformers for text to transformers for images, showing how to represent visual information as sequences of tokens, which is an essential concept for subsequent chapters.
The roadmap places ViT image classification at the bridge between text transformers and later image-generation systems. section 3.0
