Chapter 8: Image classification

Introduction

The chapter introduction places ConvNets in the history of deep learning’s success in computer vision and describes the widespread use of vision models today.

• ConvNets in computer vision: Explain that convolutional neural networks, or ConvNets, are used by most computer vision applications, and that deep learning–based vision models appear in familiar products as well as active research areas.
  Why it matters: This establishes why ConvNets are worth studying for image classification.
• Early success and delayed adoption: Trace how ConvNet results in image classification competitions helped launch deep learning’s rise, while explaining why those early successes did not immediately make it mainstream in computer vision.
  Why it matters: It shows that strong results and broad acceptance did not arrive at the same time.

Key terms: convolutional neural networks, ConvNets, CNNs, computer vision, image classification

8.1 Introduction to ConvNets

Introduces a small ConvNet for MNIST by tracing its image input through feature maps to a classifier and comparing its test accuracy with a densely connected model.

• ConvNet Input and Layer Stack: Describe the example ConvNet as a stack of Conv2D and MaxPooling2D layers that takes MNIST images with shape (28, 28, 1), excluding the batch dimension. Distinguish the channels-last shape used here from the channels-first convention used by some frameworks.
  Why it matters: The input shape and layer stack establish what data this example processes before the individual layers are explained.
• Changing Feature Map Shapes: Trace the shapes produced by Conv2D and MaxPooling2D layers: their outputs remain 3D height-width-channels tensors, height and width tend to shrink deeper in the model, and the Conv2D first argument controls the channel count.
  Why it matters: Following these shapes makes the transition from image-like feature maps to the final classifier intelligible.
• Feature Maps to Classification: Explain how GlobalAveragePooling2D turns the final (3, 3, 256) feature-map tensor into a (256,) vector for Dense layers, and describe the 10-output softmax and sparse categorical crossentropy choices for integer-labeled MNIST classification.
  Why it matters: This explains how the image-processing stack connects to a familiar classifier and why its output and loss fit the task.
• ConvNet Test Accuracy Comparison: State that the basic ConvNet reaches 99.1% test accuracy on MNIST versus 97.8% for the earlier densely connected model, reducing the error rate by about 60% relative.
  Why it matters: The comparison motivates examining what the ConvNet layers do without claiming their mechanism has already been explained.

Key terms: ConvNet, Conv2D, MaxPooling2D, image_channels, channels_first, feature map, channels, GlobalAveragePooling2D, Dense, softmax

8.1.1 The convolution operation

This section explains how convolutions learn reusable local image patterns, produce feature maps, and use padding and strides to control output size.

• Local patterns and spatial hierarchy: Distinguish the local patterns learned by convolution layers from the global patterns learned by dense layers, and explain why local pattern learning suits images.
  Why it matters: These properties explain why ConvNets can learn useful image representations efficiently.
• Feature maps and filter responses: Describe the axes of input and output feature maps and what an output channel represents.
  Why it matters: Reading output channels as spatial filter responses makes ConvNet representations easier to interpret.
• Sliding windows and shared kernel: Trace how a convolution turns input patches into an output feature map and identify its two key layer parameters.
  Why it matters: This connects a Conv2D layer's settings to what it computes at each image location.
• Border effects and padding: Explain why an unpadded convolution can shrink a feature map and how padding changes the available window locations.
  Why it matters: Padding explains why a convolution's output may shrink or retain the input's spatial size.
• Strides and spatial downsampling: Describe what convolution stride controls and how a stride greater than one affects output size.
  Why it matters: Stride is the other output-size setting needed to make sense of a convolution's spatial dimensions.

Key terms: dense layers, convolution layers, local patterns, translation invariant, spatial hierarchies, feature map, spatial axes, depth axis, channels, filters

8.1.2 The max-pooling operation

This section explains how max pooling downsamples ConvNet feature maps, why downsampling supports spatial feature hierarchies, and why max pooling is often preferred to other downsampling methods.

• Max-pooling operation: Describe how max pooling turns each local window of a feature map into the maximum value for each channel, typically using 2 × 2 windows with stride 2 to halve its spatial dimensions.
  Why it matters: This is the operation behind the shrinking feature maps in the ConvNet example.
• Purpose of spatial downsampling: Explain why reducing feature-map size helps successive convolution layers represent patterns spanning more of the original image, rather than retaining large, spatially dispersed maps throughout the network.
  Why it matters: Shrinking maps lets later features reflect more of the image instead of remaining confined to small regions.
• Max pooling versus alternatives: Explain why the section favors dense feature maps followed by max pooling over strided convolution or average pooling for preserving evidence that a feature is present.
  Why it matters: The choice of downsampling method affects whether strong evidence for a visual feature is retained.

Key terms: max pooling, feature maps, MaxPooling2D, stride, downsampling, spatial hierarchy of features, channels, average pooling, strided convolutions, feature presence

8.2 Training a ConvNet from scratch on a small dataset

This section introduces the practical problem of classifying images with limited data, using a cats-versus-dogs dataset as its example.

• Small-data classification setting: Describe why image classification with limited data is a practical problem and identify the cats-versus-dogs dataset used as the example, including its training, validation, and test split.
  Why it matters: The dataset setup makes clear what “small data” means in this example before the model is built.

Key terms: image-classification model, small datasets, training, validation, testing

8.2.1 The relevance of deep learning for small-data problems

This section explains why small image datasets can sometimes support useful ConvNets and how pretrained models offer another way to work with little data.

• Small-data ConvNet viability: Explain why there is no fixed number of samples that is always enough, and why a small, well-regularized ConvNet can sometimes give reasonable results on a simple image task with only a few hundred samples.
  Why it matters: It helps you judge whether training an image model from scratch is plausible with limited data.
• Pretrained feature reuse: Explain how reusing a model trained on a large dataset can help build a vision model when little data is available.
  Why it matters: It identifies an alternative to learning all the features of a vision model from a small dataset.

Key terms: enough samples, ConvNet, local, translation-invariant features, well regularized, custom feature engineering, pretrained classification models, bootstrap, feature reuse

8.2.2 Downloading the data

This section shows how to obtain the Dogs vs. Cats images and create a small, balanced training, validation, and test dataset.

• Kaggle dataset acquisition: Describe how to obtain the Dogs vs. Cats competition data in Colab and identify which downloaded archive is used.
  Why it matters: It identifies the source and archive needed to reproduce the example.
• Small dataset split: Describe the sizes and purposes of the three subsets made from the original images, and explain why the example deliberately uses a small sample.
  Why it matters: The deliberately limited split makes this example practice for image problems with scarce data.
• Balanced classes and accuracy: Explain why classification accuracy is an appropriate measure of success for the prepared dataset.
  Why it matters: The class balance explains the section’s choice of accuracy as a success measure.

Key terms: Dogs vs. Cats dataset, Kaggle API key, kagglehub, train.zip, test1.zip, training set, validation set, test set, subsampled dataset, balanced binary classification problem

8.2.3 Building your model

This section builds a larger ConvNet for dogs-versus-cats classification and explains its feature-map dimensions, input scaling, and binary-classification setup.

• Larger ConvNet architecture: Explain why this ConvNet adds two Conv2D–MaxPooling2D stages to the earlier pattern, and describe how feature-map size and depth change through the model.
  Why it matters: It connects the model’s added layers to both task complexity and manageable feature-map dimensions.
• Binary classification output and loss: Explain how the output layer represents the dogs-versus-cats prediction and why binary crossentropy is chosen when compiling the model.
  Why it matters: The output and loss must fit the fact that the model is choosing between two classes.
• Input pixel rescaling: Describe what the model’s initial Rescaling layer does to image inputs.
  Why it matters: It identifies an input transformation built into this model rather than left implicit.

Key terms: Conv2D, MaxPooling2D, feature maps, capacity, GlobalAveragePooling2D, binary classification, Dense, sigmoid, binary crossentropy, adam optimizer

8.2.4 Data preprocessing

This section turns image files into batched datasets for a ConvNet, trains and checkpoints the model, and interprets its validation and test results as evidence of overfitting.

• Image file preprocessing pipeline: Trace how JPEG files in class subdirectories become batches of image tensors and labels using image_dataset_from_directory.
  Why it matters: The model needs consistently sized tensor batches rather than JPEG files on disk.
• Dataset iteration and transformations: Describe what a tf.data.Dataset yields and how its methods support batching and preprocessing before fitting a Keras model.
  Why it matters: Datasets provide a reusable way to prepare and deliver data without writing the entire input pipeline yourself.
• Validation based model checkpoint: Explain how fit() uses a separate validation Dataset and how ModelCheckpoint preserves the best epoch according to validation loss.
  Why it matters: Saving the best validation epoch lets you evaluate the model before later training makes overfitting worse.
• Overfitting in training curves: Describe the contrasting training and validation trends, and explain why the saved model is used for test evaluation.
  Why it matters: The gap between training and validation performance warns against treating near-perfect training accuracy as success on new images.

Key terms: image_dataset_from_directory, tf.data.Dataset, RGB, batches, batch(), shuffle(), prefetch(), map(), fit(), validation_data

8.2.5 Using data augmentation

This section uses random image transformations and dropout to reduce overfitting when training a ConvNet on a small dataset.

• Purpose of data augmentation: Explain how data augmentation creates varied training inputs from existing images and why that can help a model generalize.
  Why it matters: This explains why varying a small training set can help with overfitting.
• Augmentation layers and placement: Describe the two places Keras augmentation layers can be applied, their stated processing difference, and examples of the transformations used.
  Why it matters: Where augmentation runs affects whether this example uses model-side GPU work or pipeline-side CPU work.
• Augmentation limits and dropout: Explain why augmentation alone may not eliminate overfitting and why the example adds dropout.
  Why it matters: More variations do not remove the fundamental shortage of original data.
• Training-only augmentation and results: State when augmentation is applied and describe the reported change in overfitting and accuracy when training with augmentation and dropout.
  Why it matters: The results show what the combined regularization changes—and why evaluation uses unaugmented images.

Key terms: data augmentation, random transformations, generalize, data augmentation layers, Dataset, map(), RandomFlip, RandomRotation, RandomZoom, intercorrelated

8.3 Using a pretrained model

This section explains why a pretrained image model can help with a small image dataset and introduces ImageNet-trained Xception as an example.

• Transferable pretrained visual features: Explain what a pretrained model is and why its learned features can help on a small image dataset, even when the new task has different classes.
  Why it matters: It explains why a small dataset need not be the only source of useful visual features.
• ImageNet model choice: Justify why an ImageNet-trained ConvNet is a reasonable pretrained model to consider for dogs-versus-cats classification.
  Why it matters: The example connects the general case for pretrained features to this chapter’s classification task.

Key terms: pretrained model, spatial hierarchy of features, small-data problems, ImageNet, ConvNet, dogs-versus-cats classification

8.3.1 Feature extraction with a pretrained model

This section shows how to reuse a pretrained ConvNet’s features for a new image classifier, comparing a fast feature-recording approach with a slower approach that supports data augmentation.

• Reusable convolutional base: Explain how feature extraction uses a pretrained ConvNet’s convolutional base to produce features for a new classifier, and why the original classifier is generally not reused.
  Why it matters: It explains which part of a pretrained image model is usually worth carrying over to a new classification task.
• Feature generality by depth: Distinguish the features learned by early and later convolutional layers, and explain when using only early layers may be preferable.
  Why it matters: The best point at which to extract features depends on how similar the new images are to the pretrained model’s data.
• Matching pretrained image preprocessing: Explain why input images must match a pretrained ConvNet’s expected preprocessing and how the section obtains the matching conversion.
  Why it matters: Matching preprocessing lets the pretrained feature extractor work with inputs in the form it learned from.
• Feature extraction workflow tradeoff: Compare recording pretrained features once with training a model that includes the convolutional base, including their cost and ability to use data augmentation.
  Why it matters: The choice determines both training cost and whether augmentation can help with overfitting on a small image dataset.
• Freezing the pretrained base: Explain why and how the convolutional base is frozen when a new dense classifier is trained with data augmentation.
  Why it matters: Freezing preserves useful pretrained features while the newly added classifier learns the new task.

Key terms: feature extraction, convolutional base, backbone, feature maps, densely connected classifier, convolution layers, KerasHub, ImageConverter, from_preset(), pretrained checkpoint

8.3.2 Fine-tuning a pretrained model

This section explains how to fine-tune a pretrained image classifier safely, including which layers to train and why to use a low learning rate.

• Fine-tuning and feature extraction: Explain how fine-tuning extends feature extraction by unfreezing a pretrained base and training it together with the added classifier to adapt its representations to the new problem.
  Why it matters: This distinction explains what fine-tuning changes beyond using a pretrained base for feature extraction.
• Safe fine-tuning sequence: Trace the order of adding a classifier, freezing the pretrained base, training the classifier, unfreezing the base, and training them jointly. Explain why the classifier must be trained before the base is unfrozen.
  Why it matters: The training order protects useful pretrained representations while the new classifier is learning.
• Selective upper-layer fine-tuning: Justify fine-tuning only some top layers of a large pretrained convolutional base while leaving lower layers frozen, especially with a small dataset.
  Why it matters: This trade-off helps you adapt a large model without needlessly training its many parameters on limited data.
• Small fine-tuning updates: Explain why fine-tuning uses a very low learning rate.
  Why it matters: Small updates let the model adapt without damaging features it has already learned.

Key terms: fine-tuning, feature extraction, model base, fully connected classifier, freeze, unfreeze, convolutional base, error signal, BatchNormalization, partial fine-tuning

8.4 Summary