Chapter 7: Telling birds from airplanes: Learning from images

Introduction

The introduction shifts from a simple regression example to an upcoming image-recognition task using tiny images.

7.1 A dataset of tiny images

Introduces CIFAR-10 as a small labeled image dataset that is useful for learning image recognition, despite being too simple for new research.

• CIFAR-10 images and labels: Describe the size and format of CIFAR-10 images and how their class labels are represented.
  Why it matters: The image shape and class labels define the data used in the chapter’s recognition task.
• CIFAR-10 as learning material: Explain why the section uses CIFAR-10 even though it is not considered suitable for developing or validating new research.
  Why it matters: The dataset is chosen to make the ideas approachable, not to benchmark new research.

Key terms: CIFAR-10, RGB, classes

7.1.1 Downloading CIFAR-10

This section shows how to download CIFAR-10 with torchvision and identifies the common Dataset base class used by torchvision datasets.

• CIFAR-10 download arguments: Describe how to use torchvision.datasets.CIFAR10 to obtain the data and what its three arguments specify.
  Why it matters: These choices determine which CIFAR-10 data you obtain and whether missing data can be fetched automatically.
• Torchvision dataset base class: Explain what torchvision’s datasets submodule provides beyond CIFAR-10 and what base class its returned datasets share.
  Why it matters: CIFAR-10 is one example of the dataset objects provided through a shared PyTorch base class.

Key terms: torchvision, datasets module, CIFAR10, training set, validation set, datasets submodule, torch.utils.data.Dataset, subclass

7.1.2 The Dataset class

This section explains how a PyTorch Dataset exposes its items and shows what an item from CIFAR-10 looks like.

• Dataset access methods: Explain what __len__ and __getitem__ provide in a PyTorch Dataset, how ordinary Python code uses them, and what an item from the CIFAR-10 example contains.
  Why it matters: These methods let you count and retrieve image examples using familiar Python operations.

Key terms: torch.utils.data.Dataset, __len__, __getitem__, sample, label, CIFAR10

7.1.3 Dataset transforms

This section shows how to use torchvision dataset transforms to turn CIFAR-10 images into tensors and explains the resulting format.

• Transforms in the dataset pipeline: Explain how a torchvision transform can be passed to a dataset so that it changes an image after loading but before the dataset returns it.
  Why it matters: This explains why retrieving an item from the transformed dataset gives you a tensor instead of a PIL image.
• ToTensor image conversion: Describe how ToTensor changes a PIL image or NumPy array into a tensor, including its dimension order, numeric type, and value range.
  Why it matters: The tensor’s shape and values determine what the model receives as image data.
• Channel order for display: Explain why displaying a ToTensor image with Matplotlib requires permuting its axes.
  Why it matters: An image tensor can need a different axis order for display than for use in PyTorch.

Key terms: torchvision.transforms, transform, __getitem__, datasets.CIFAR10, ToTensor, C × H × W, float32, label, permute, H × W × C

7.1.4 Normalizing data

Explains why CIFAR-10 image channels are normalized, how to compute and apply the normalization, and why normalized images may look wrong when plotted.

• Reasons for channel normalization: Explain why giving each image channel zero mean and unit standard deviation can help a neural network learn.
  Why it matters: Normalization helps the channels contribute on comparable terms during training.
• Computing and applying normalization: Describe how the CIFAR-10 training set supplies per-channel statistics and how transforms.Normalize uses them after ToTensor.
  Why it matters: The transform needs statistics from the training images before it can normalize each tensor.
• Appearance of normalized images: Explain why a normalized CIFAR-10 image can look black when plotted without implying that its image data was lost.
  Why it matters: A misleading plot does not mean normalization destroyed the image.

Key terms: zero mean, unitary standard deviation, nonzero gradients, learning rate, transforms.Normalize, transforms.Compose, ToTensor, mean, stdev, view(3, -1)

7.2 Distinguishing birds from airplanes

This section introduces a bird-watching blog’s need to automatically reject airplane photos instead of having Jane delete them manually.

• Automated airplane photo filtering: Explain why Jane needs a neural network to distinguish birds from airplanes in the camera images and what it should do with airplane photos.
  Why it matters: The filtering task gives the image classifier a concrete purpose.

Key terms: birds, airplanes, neural network

7.2.1 Building the dataset

This section builds a two-class birds-versus-airplanes dataset from CIFAR-10 and explains why the filtered object can be used as a dataset without subclassing PyTorch’s Dataset class.

• Two Class Dataset Construction: Describe how the section creates a dataset for the classifier by filtering CIFAR-10 to birds and airplanes and remapping their labels to contiguous values.
  Why it matters: This prepares the examples and labels for the birds-versus-airplanes classifier.
• Dataset Interface Shortcut: Explain why the filtered cifar2 object can serve as a dataset without being a Dataset subclass, and state the limitation of that shortcut.
  Why it matters: You can recognize why the simple filtered object works here without treating the shortcut as a universal replacement for a Dataset class.

Key terms: cifar10, cifar2, remap the labels, contiguous, Dataset, __len__, __getitem__

7.2.2 A fully connected model

The section builds a fully connected image classifier by flattening pixels, then explains why it needs a nonlinear hidden layer and what spatial information its structure misses.

• Flattened image classifier design: Describe how a 32 × 32 color image becomes 3,072 input features and how two linear layers reduce those features to two outputs for the two classes.
  Why it matters: This gives the image data a shape the fully connected model can accept.
• Nonlinear hidden feature layer: Explain why the model puts an activation between two linear layers and what its learned hidden features represent.
  Why it matters: The activation is what makes this more than a sequence of linear mappings.
• Missing spatial awareness: Explain why flattening allows the model to compare pixel values but gives it no built-in awareness of which pixels are neighbors.
  Why it matters: Flattening makes the image usable as features, but it does not build the image’s spatial layout into the model.

Key terms: 1D vector, input features, nn.Linear, hidden features, activation, nonlinearity, weight matrix, spatial configuration, structurally unaware, adjacent

7.2.3 Output of a classifier

This section explains why a bird-versus-airplane classifier should produce categorical, probability-like outputs rather than a single regression value.

• Categorical versus regression outputs: Distinguish a classifier’s categorical output from the quantitative temperature prediction in chapter 6, and explain why treating airplane and bird as numeric targets for MSELoss is a poor fit.
  Why it matters: The kind of value being predicted determines how the classifier’s output should be represented.
• Class vectors as probabilities: Describe one-hot class targets and explain how to interpret an imperfect classifier’s two output entries as probabilities, including their required range and sum.
  Why it matters: This interpretation makes the classifier’s output meaningful even when it is not perfectly certain.
• Binary single probability alternative: Explain why two probability outputs are redundant for binary classification and state the single-output PyTorch alternative named in the section.
  Why it matters: A two-class classifier need not emit two independent probabilities.

Key terms: categorical, regression problem, MSELoss, one-hot encoding, probabilities, softmax, binary classification, nn.Sigmoid, nn.BCELoss, nn.BCELossWithLogits

7.2.4 Representing the output as probabilities

The section turns a bird-versus-airplane classifier’s scores into probabilities, explains how labels and classification losses give those outputs meaning, and shows the corresponding PyTorch loss choices.

• Softmax probability conversion: Explain how softmax converts a vector of scores into probabilities, and describe what it preserves and changes about the scores.
  Why it matters: It explains how the classifier’s raw scores become comparable class probabilities.
• Batched classification outputs: Describe how softmax and class selection operate on batched classifier outputs, and explain why an untrained prediction has no established class meaning.
  Why it matters: A probability pair is useful as a prediction only when its entries are tied to the labeled classes.
• Correct-class likelihood loss: Explain why classification loss focuses on the probability of the correct class and describe how negative log likelihood is computed.
  Why it matters: The loss connects each labeled example to the probability the model should increase.
• Stable PyTorch loss inputs: Distinguish the inputs to nn.NLLLoss and cross-entropy loss, and explain the role of nn.LogSoftmax.
  Why it matters: Passing the right form of output to the loss avoids a mistaken or numerically fragile training setup.
• Classification gradient comparison: Explain the section’s comparison of cross-entropy and MSE gradients for classification.
  Why it matters: A responsive gradient helps the classifier keep adjusting its predictions during training.

Key terms: softmax, probabilities, monotone, scale invariant, nn.Softmax, dimension 1, class_names, argmax, torch.max, class indices

7.2.5 Training the classifier

This section trains a bird-versus-airplane classifier with minibatches, evaluates its generalization, and shows why a fully connected model scales poorly to larger images.

• Minibatch gradient updates: Explain how updating from shuffled minibatches differs from accumulating a gradient over the whole dataset, and why the section uses a reasonably small learning rate.
  Why it matters: This explains why the training loop updates from small groups of images rather than waiting for all 10,000.
• DataLoader minibatch delivery: Describe how DataLoader supplies shuffled minibatches from a Dataset and what its image and label tensors contain in this example.
  Why it matters: DataLoader lets the training loop work with minibatches without manually selecting and stacking images.
• Classifier loss input conventions: Distinguish the outputs expected by nn.NLLLoss and nn.CrossEntropyLoss, and explain what changes when LogSoftmax is removed from the model.
  Why it matters: The choice of loss determines what the network should output and how to interpret its predictions.
• Validation accuracy and overfitting: Explain why the classifier is assessed by validation accuracy and what the gap between training and validation accuracy reveals.
  Why it matters: Falling loss or perfect training accuracy alone does not show that the classifier works on independent images.
• Fully connected parameter scaling: Explain why the first linear layer dominates the parameter count and why this makes the classifier impractical for much larger images.
  Why it matters: The parameter count exposes a practical limit of applying fully connected layers directly to image pixels.

Key terms: batch, minibatches, stochastic gradient descent, epoch, learning rate, DataLoader, Dataset, batch_size, shuffle, imgs

7.2.6 The limits of going fully connected

Explains why a fully connected classifier can learn from images yet struggle to generalize across changes in an object’s position.

• Flattened Images and Spatial Relationships: Explain what a linear module does with a flattened RGB image and why that structure does not take advantage of pixel positions.
  Why it matters: The model has access to every pixel without making use of the image’s spatial structure.
• Position-Dependent Airplane Recognition: Describe why shifting the same airplane to another position creates a problem for the fully connected classifier.
  Why it matters: An object’s identity does not change when it moves within an image, but this classifier does not automatically carry recognition across positions.
• Augmentation Capacity and Overfitting: Explain why training on randomly translated images is a costly workaround and how the model’s structure contributes to overfitting.
  Why it matters: Seeing more positions during training does not remove the cost of a model that must learn them separately.

Key terms: linear module, fully connected network, relative position, translation invariant, data augmentation, random translations, hidden features, parameters, overfitting

7.3 Conclusion

The chapter concludes by summarizing its image-classification workflow, identifying the limits of treating images as one-dimensional data, and noting where the same network design may still be useful.

• Classification Workflow Components: Describe the chapter’s classification workflow as moving from a dataset to a model and minimizing an appropriate loss in a training loop.
  Why it matters: This workflow is a foundation for later PyTorch classification projects.
• Image Layout and Translation Limitation: Explain why the chapter’s model has a severe limitation for images: it treats two-dimensional images as one-dimensional data and has no natural way to incorporate translation invariance.
  Why it matters: It explains why this working classifier is not yet well suited to image structure.
• Applications Beyond Shift Invariant Data: Describe where the chapter’s network design may be useful without image-style translation invariance, and distinguish bag-of-words text from models that account for word positions.
  Why it matters: The image model’s limitation does not make the workflow useless for other kinds of data.

Key terms: dataset, model, loss, training loop, 2D images, 1D data, translation invariance, tabular data, time-series data, bag-of-words models

7.4 Exercises

7.5 Summary