Chapter 13: Training a classification model to detect suspected tumors

Introduction

The chapter introduction connects the previously prepared CT-scan dataset to the upcoming work of loading data, training a classifier, and tracking its performance.

13.1 A foundational model and training loop

This section defines the nodule-classification task and outlines a training-and-validation loop that will provide an evaluable foundation for the larger project.

• Candidate classification target: Explain that this classifier assigns each candidate one label: nodule or non-nodule, rather than deciding whether a nodule is malignant.
  Why it matters: The label being predicted determines what the model’s results mean.
• Training and validation flow: Trace the outlined epoch: initialize the model and data loading, train on batches by predicting, calculating loss, recording metrics, and updating weights, then evaluate validation batches and report progress.
  Why it matters: Separating weight updates from validation measurements makes the reported training progress interpretable.
• Value of an evaluable baseline: Justify building an early end-to-end version that can be evaluated before extensive experimentation.
  Why it matters: A measurable starting point helps you tell whether later tinkering actually helps.
• Project structure tradeoff: Explain why this more complex project uses contained functions and separate modules, while avoiding unnecessary infrastructure.
  Why it matters: Appropriate structure supports experimentation without making infrastructure the project.
• Initial performance measurements: Describe which metrics this chapter will collect and why choosing suitable metrics matters.
  Why it matters: Measurements determine what you can—and cannot—conclude about the classifier’s progress.

Key terms: candidate, nodule, non-nodule, classification, epoch, batch, loss, backpropagation, training, validation

13.2 The main entry point for our application

The section structures training as a command-line application that can also be instantiated and run directly from Python or Jupyter.

• Reusable training application: Explain why the training application is implemented as a class and how that supports both shell and notebook use.
  Why it matters: The same training application can be convenient to run from a shell and to explore interactively.
• Configuration and execution entry points: Trace how the application starts, accepts arguments, and separates configuration from its core logic.
  Why it matters: Separating argument-based setup from main makes the application's entry point reusable.

Key terms: command-line application, application class, Jupyter Notebook, native CLI, if main stanza, argparse, __init__, main

13.3 Pretraining setup and initialization

The section identifies what must be initialized before a training epoch can process batches and explains the roles of LunaDataset and DataLoader.

• Preloop training initialization: Explain which components must be initialized before iterating over batches and what LunaDataset and DataLoader each contribute.
  Why it matters: These components must be in place before the application can receive batches for training.

Key terms: model, optimizer, LunaDataset, DataLoader, training epoch

13.3.1 Initializing the model and optimizer

Explains how the training application sets up GPU use and chooses an initial optimizer without tuning the model yet.

• Multi-GPU wrapper choice: Explain why the application uses nn.DataParallel when multiple GPUs are available and how it differs from DistributedDataParallel at the level described here.
  Why it matters: The wrapper choice keeps this training setup simple while setting a limit on how it can scale.
• Model placement before optimization: Explain why the application moves the model to the GPU before constructing the optimizer when CUDA is used.
  Why it matters: The order of initialization determines whether the optimizer tracks the parameters the GPU model will use.
• Starting optimizer configuration: Justify the initial choice of SGD with momentum and its settings, and describe why hyperparameter search is deferred.
  Why it matters: A workable baseline lets the project address larger problems before fine-tuning settings.

Key terms: nn.DataParallel, DistributedDataParallel, self.model.to(device), model parameters, optimizer, GPU, stochastic gradient descent, SGD, momentum, learning rate

13.3.2 Care and feeding of data loaders

This section explains how LunaDataset samples become batched, five-dimensional inputs for a 3D convolutional model, and how DataLoader workers can keep training supplied with data.

• CT channel and batch dimensions: Explain how a CT sample gains channel and batch dimensions to meet the input shape expected by torch.nn.Conv3d.
  Why it matters: These dimensions let CT data fit the model’s expected input without mistaking channels for separate scans.
• DataLoader batch collation: Describe why samples are grouped into batches and what DataLoader does with LunaDataset samples.
  Why it matters: Batching uses available processing capacity without requiring the application to assemble each batch itself.
• Parallel data loading: Explain how DataLoader workers prepare batches while the model processes other data.
  Why it matters: Background loading can reduce the time the GPU spends waiting for data.

Key terms: LunaDataset, torch.nn.Conv3d, channel, batch, (N, C, D, H, W), ct_t.unsqueeze(0), DataLoader, sample tuples, batch tuple, collation

13.4 Our first-pass neural network design

The section motivates a pragmatic first-pass 3D classification network based on established image-recognition designs.

• Pragmatic starting architecture: Explain why the first tumor-detection network starts from a tested image-recognition design rather than seeking an optimal architecture immediately, and why that design needs adaptation for 3D input.
  Why it matters: This explains why the project begins with an adaptable baseline instead of trying to find the best possible network at once.

Key terms: 2D images, 3D, first network architecture

13.4.1 The core convolutions

This section explains the tail–backbone–head structure of the classifier and how its repeated convolutional blocks process 3D CT data.

• Classifier parts and data flow: Describe how data moves from the tail through the backbone to the head, and what each part does in this classifier.
  Why it matters: This gives you a way to explain where each part of the classifier fits.
• Repeated block design: Describe the layers in a LunaBlock and explain why the authors begin with a relatively simple network design.
  Why it matters: The repeated block is the main building unit of this model’s backbone.
• Stacked convolution receptive fields: Explain how stacking two 3 × 3 × 3 convolutions expands the input region that can affect an output voxel, and why this is preferable here to one larger convolution.
  Why it matters: It explains how the block considers a wider area without using one larger convolution.
• Pooling and padded output sizes: Describe what the block’s max pool does and distinguish its size-reducing effect from the size-preserving padded convolutions.
  Why it matters: It clarifies which operation reduces spatial size and why discarded voxels may still affect the result.

Key terms: tail, backbone, head, LunaBlock, convolutions, activation, max-pooling, receptive field, 3 × 3 × 3 convolution, 5 × 5 × 5 convolution

13.4.2 The full model

This section completes the LunaModel by explaining its input normalization, four-block backbone, transition to a classification head, two output forms, and weight initialization.

• Input scale normalization: Explain how the model normalizes CT input and what that means for its original Hounsfield-unit scale.
  Why it matters: It explains why later layers see normalized values rather than raw CT intensities.
• Backbone resolution reduction: Trace how the four repeated backbone blocks change the spatial size of a CT chunk.
  Why it matters: The backbone’s output size determines what the classification head must accept.
• Convolution to linear transition: Explain why and how the model reshapes the backbone output before its fully connected layer.
  Why it matters: Flattening connects spatial features from the backbone to the classifier without adding learned parameters.
• Logits and classification probabilities: Distinguish the model’s raw logits from its softmax probabilities and explain why it returns both.
  Why it matters: The two outputs serve different purposes even though they come from the same classifier.
• Stable parameter initialization: Justify initializing network weights so intermediate outputs and gradients stay well behaved.
  Why it matters: Initialization helps keep signals and gradients at workable scales as they pass through the network.

Key terms: nn.BatchNorm3d, Hounsfield unit (HU), backbone, 2 × 2 × 2 max-pool, self.block4, fully connected layer, view, flatten, forward, logits

13.5 Training and validating the model

This section shows how the training loop assembles the model and data-loading pieces, while tracking sample-level metrics and organizing batch processing.

• Per-sample training metrics: Explain that trnMetrics_g collects detailed per-class training metrics and carries information about the model’s behavior on each sample from computeBatchLoss to logMetrics.
  Why it matters: It explains how the training loop gathers the information needed to inspect model behavior, not just compute a loss.
• Training loop organization: Describe why the loop uses tqdm around the training data loader and places loss computation in computeBatchLoss: tqdm displays processing speed and estimated completion time, while the separate method supports code reuse.
  Why it matters: These choices make a familiar training loop easier to monitor and its loss computation easier to reuse.

Key terms: trnMetrics_g, computeBatchLoss, logMetrics, per-class metrics, tqdm, train_dl

13.5.1 The computeBatchLoss function

Explains how computeBatchLoss returns a batch loss for training while retaining per-sample results for evaluating and debugging the classifier.

• Per-sample loss and batch mean: Explain why computeBatchLoss obtains a CrossEntropyLoss value for each sample and returns their mean. Describe how this supplies the batch loss while preserving individual losses for later aggregation.
  Why it matters: This distinction explains how the same calculation supports both weight updates and more detailed analysis.
• Per-sample metrics for diagnosis: Describe what computeBatchLoss records for each sample and how those records help assess performance and investigate model problems.
  Why it matters: Per-sample records can reveal trouble that a single batch loss would hide.

Key terms: computeBatchLoss, CrossEntropyLoss, per-sample losses, batch loss, metrics_g, label, prediction, loss, per-class statistics

13.5.2 The validation loop is similar

Explains how validation reuses the batch-loss calculation while leaving the model unchanged and collecting metrics.

• Read-only validation loop: Explain why validation does not update model weights and how `torch.no_grad()` fits that read-only loop.
  Why it matters: Validation can measure the current model only if it does not change that model.
• Metrics despite unused loss: Describe what the validation loop still obtains from `computeBatchLoss` even though it ignores the returned loss.
  Why it matters: Ignoring the returned loss does not mean the validation loop stops gathering performance information.

Key terms: validation, torch.no_grad(), optimizer, weights, computeBatchLoss, valMetrics_g, per-batch loss

13.6 Outputting performance metrics

This section explains what the training script logs after each epoch, where the results come from, and why monitoring them matters.

• Purpose of progress logging: Explain why the script logs performance metrics after each epoch and then continues training.
  Why it matters: Metrics make it possible to spot unproductive training rather than letting it run unnoticed.
• Per-epoch metric sources: Describe which collected results supply the reported training and validation metrics, and name the metrics they support.
  Why it matters: Keeping training and validation results available lets the script report both sets of metrics.

Key terms: performance metrics, does not converge, epoch, trnMetrics_g, valMetrics_g, percent correct, average loss per class

13.6.1 The logMetrics function

The section explains how logMetrics uses per-sample tensors and binary masks to calculate and log overall and per-class training or validation results.

• Metrics tensor inputs: Describe what logMetrics receives and how it distinguishes training results from validation results.
  Why it matters: The inputs explain where the values being summarized come from.
• Binary class masks: Explain how label and prediction masks separate nodule from non-nodule samples and why this masking logic is specific to binary classification.
  Why it matters: The masks make it possible to see results separately for nodules and non-nodules.
• Per-class performance summaries: Describe the loss and correctness statistics logMetrics calculates and how it reports them.
  Why it matters: An overall result alone can hide poor performance on one of the two classes.

Key terms: logMetrics, metrics_t, mode_str, epoch_ndx, classificationThreshold, negLabel_mask, negPred_mask, positive masks, negative masks, metrics_dict

13.7 Running the training script

This section starts the training workflow and shows how to recognize cache preparation delays and check whether the GPU is being kept busy.

• Training run prerequisites: State where to run the training script and what must be installed before starting it.
  Why it matters: The expected directory and dependencies let the training application run.
• First epoch cache delay: Explain why the first epoch may be unusually slow and how the section suggests preparing the cache.
  Why it matters: A slow first epoch may be data preparation rather than slow model computation.
• CPU and GPU bottleneck checks: Describe how to use CPU and GPU utilization during training to check for a data-loading bottleneck or GPU saturation, including the limits of the CPU clue.
  Why it matters: These checks help distinguish time spent supplying batches from time spent using the GPU.

Key terms: main code directory, requirements.txt, LunaDataset, prepcache.py, dsets.py, cache, top, nvidia-smi, GPU-Util, CPU bottlenecks

13.7.1 Data needed for training

This subsection shows how to check that the expected training and validation data and underlying LUNA files are present before troubleshooting training.

• Expected dataset sample counts: Explain why unexpectedly low training or validation sample counts should prompt a check that the full dataset is present.
  Why it matters: A training run is hard to trust if it may be using an incomplete dataset.
• LUNA file completeness checks: Describe the file checks suggested when confirming that the LUNA data is present.
  Why it matters: These checks help distinguish missing source files from problems elsewhere in training.

Key terms: samples, training, validation, series UID, .mhd file, .raw file, data-unversioned/part2/luna

13.7.2 Interlude: The tqdm function

This section introduces tqdm as a way to track progress and estimate completion time during long-running deep learning loops.

• Tqdm progress tracking: Explain how tqdm tracks a long-running loop and what its progress display tells you about the work remaining.
  Why it matters: A useful completion estimate makes a long training run easier to manage and can reveal when it is taking unexpectedly long.

Key terms: tqdm, progress bar, time remaining, iterations per second

13.8 Evaluating the model: Getting 99.7% correct means we’re done, right?

The section shows why 99.7% accuracy is misleading when the classifier misses every nodule, and why that error matters.

• Misleading overall accuracy: Explain why at least 99.7% correct results do not mean the model detects nodules: nodules make up only about 0.3% of samples, and the network classifies everything as not-a-nodule.
  Why it matters: Overall accuracy can conceal complete failure on the cases the model is meant to find.
• Slow learning in losses: Describe what the results after 10 epochs do and do not show: no positive samples are correctly identified, although decreasing validation positive loss without a corresponding rise in negative loss suggests some very slow learning.
  Why it matters: The loss offers a more nuanced reading of training progress than the unchanged classifications alone.
• Consequences of missed tumors: Explain why classifying a tumor as an innocuous structure is especially dangerous and why misclassification consequences should inform model design, training, and evaluation.
  Why it matters: The cost of missing a tumor makes the model’s apparent accuracy an unsafe basis for declaring success.

Key terms: validation set, non-nodules, nodules, not-a-nodule, val_pos loss, val_neg loss, positive samples, misclassification, tumor, innocuous structure

13.9 Graphing training metrics with TensorBoard

The section explains why graphing training metrics in TensorBoard is useful and why it can be used with PyTorch.

• Visualizing training metric trends: Explain how graphing training metrics helps interpret model training compared with looking only at values from individual epochs.
  Why it matters: A graph gives you context for a metric value that a single epoch’s number cannot.
• TensorBoard and PyTorch compatibility: Explain why a tool that originated with TensorFlow can be used to display metrics from PyTorch.
  Why it matters: TensorBoard’s TensorFlow origins do not prevent you from using it in a PyTorch project.

Key terms: TensorBoard, training metrics, trends, outlier, TensorFlow, PyTorch APIs

13.9.1 Running TensorBoard

This section shows how to launch TensorBoard, inspect training and validation metrics, and manage which runs appear in its graphs.

• Launching the metrics dashboard: Describe how to open TensorBoard for the training script’s metrics: point its --logdir at the runs/ directory and open the dashboard in a browser.
  Why it matters: Pointing TensorBoard at the right directory lets you see the metrics from your training runs.
• Reading graphed training metrics: Explain what the Scalars graphs display and how to check that their values correspond to the training script’s printed metrics.
  Why it matters: Checking plotted values against the training output makes the graphs easier to interpret confidently.
• Controlling visible runs and smoothing: Describe how run selection and smoothing affect the readability of TensorBoard graphs, including the risk of excessive smoothing.
  Why it matters: These controls make comparisons easier without mistaking a smoothed view for the whole story.

Key terms: TensorBoard, runs/, --logdir, localhost:6006, Scalars, training and validation metrics, Value column, tooltip, smoothing, runs

13.9.2 Adding TensorBoard support to the metrics logging function

The section adds TensorBoard logging to the metrics function, explains how scalar metrics become charts, and chooses training samples rather than epochs for the chart’s X-axis.

• Separate lazy TensorBoard writers: Explain why the application creates separate SummaryWriter objects for training and validation and waits until the first metrics write to initialize them.
  Why it matters: Separate, lazily created writers keep training and validation runs distinct without cluttering TensorBoard during experiments.
• Scalar metrics as chart data: Describe how logMetrics sends the existing metrics dictionary to TensorBoard and what the add_scalar arguments mean on a chart.
  Why it matters: These arguments determine which metric appears on which chart and where each measurement appears on its axes.
• Training samples as global step: Justify using totalTrainingSamples_count as the global_step instead of the epoch number when plotting metrics.
  Why it matters: A sample-count X-axis makes comparisons less misleading when experiments use different epoch sizes.
• Noisy first epoch metrics: Explain why the first epoch’s training metrics may look noisy and describe the section’s choice about retaining them.
  Why it matters: It helps you interpret an erratic start to a metrics plot without treating it as representative performance.

Key terms: SummaryWriter, log_dir, training run, validation run, metrics_dict, add_scalar, tag, scalar_value, global_step, totalTrainingSamples_count

13.10 Why isn’t the model learning to detect nodules?

The section explains why the classifier can show high accuracy and decreasing loss without learning to detect nodules.

• Misleading Accuracy From Class Imbalance: Explain why the model can score highly while failing to detect nodules, and why decreasing loss does not overturn that concern.
  Why it matters: A strong overall score can hide a failure on the cases the detector is meant to find.

Key terms: nodule, loss, training and validation sets

13.11 Conclusion

The conclusion summarizes the working classification training pipeline and notes that its results are not yet usable.

• Current classifier training status: Describe what the chapter has put in place: a model and training loop that consume the previous chapter’s data, with metrics logged to the console and graphed. State that the results are not yet usable.
  Why it matters: It separates progress in building the training pipeline from success at producing usable results.

Key terms: model, training loop, metrics

13.12 Exercises

13.13 Summary