liveBook · Manning

Training loop

A training loop is an iterative procedure that processes training data, evaluates an objective, and updates model parameters.

How 7 Manning books explain it

Deep Learning with R, Third Edition

A training loop is a repeated process that evaluates examples, adjusts model parameters, and progressively minimizes the loss function.

How it is used
Trains models by repeatedly evaluating batches and adjusting weights to reduce loss.
How deep it goes
Explains batch steps and epochs, then implements both a simplified full loop and a stateless gradient update.
Examples
Uses MNIST handwritten-digit classification and a JAX linear classifier to illustrate built-in and custom loops.
What you can do after
Apply it via Keras fit(), or build a custom training loop from scratch for more control.

Read how Deep Learning with R, Third Edition explains training loop →

Rearchitecting LLMs

A training loop is an iterative procedure that computes model outputs, evaluates an error, and updates trainable parameters over batches and iterations.

How it is used
Recovers a pruned student model’s capabilities by iteratively aligning its behavior with a teacher.
How deep it goes
Walks through batching, forward passes, loss calculation, backpropagation, and optimizer updates, then compares recovery strategies.
Examples
Compares labels-only and feature-based distillation against teacher and pruned baselines using perplexity and capability retention at multiple data scales.
What you can do after
Apply it via PyTorch to update a student model from teacher outputs using logits-only or compound distillation losses.

Read how Rearchitecting LLMs explains training loop →

LLMs in Production

A training loop is the repeated process of computing model outputs, evaluating loss, calculating gradients, and updating parameters.

How it is used
Organizes model training from data preparation through repeated prediction, loss, backpropagation, and parameter updates.
How deep it goes
Explains the loop from batches to gradients and implementation components, then extends into speed-memory tradeoffs without deriving optimization theory.
Examples
Uses a generative CBoW model, Hamlet tokens, a GPT built from scratch, and a speed-memory comparison table.
What you can do after
Build it from scratch with modular model components, batched data, loss calculation, backpropagation, and resource-aware training choices.

Read how LLMs in Production explains training loop →

Build a DeepSeek Model (From Scratch)

A training loop is an iterative procedure that processes batches, computes predictions and loss, backpropagates gradients, updates parameters, and clears gradients.

How it is used
Runs iterative optimization that teaches models to reduce loss on batches and assesses generalization on validation data.
How deep it goes
Explains the complete update sequence, evaluation, scheduling, and checkpointing, but stops short of distributed training.
Examples
Uses a PyTorch language-model loop with TinyStories batches, validation loss, mixed precision, and best or periodic checkpoint files.
What you can do after
Build it from scratch by combining batches, model calls, loss computation, backpropagation, optimizer updates, evaluation, and checkpointing.

Read how Build a DeepSeek Model (From Scratch) explains training loop →

Fight Fraud with Machine Learning

A model training loop is an iterative procedure that updates a model using training data and evaluates its learning progress across repeated steps or epochs.

How it is used
Deploys iterative parameter updates and validation checks when training segmentation, audio, tabular, or graph-based fraud-detection models.
How deep it goes
Explains executable training and validation steps, metrics, and learning curves, but defers model architecture and broader test interpretation.
Examples
Uses receipt-image segmentation, transformer-based voice-faking detection, MLP transaction classification, and GCN modeling on the Elliptic Bitcoin dataset.
What you can do after
Build it from scratch in PyTorch, monitor training and validation metrics, and evaluate trained models on unseen data.

Read how Fight Fraud with Machine Learning explains training loop →

CUDA for LLMs

A training loop repeatedly processes training data, computes model outputs and loss, and updates model parameters.

How it is used
Establishes a verified training baseline and exposes step-level costs for comparing implementations.
How deep it goes
Traces batches through data loading, forward, loss, backward, and updates; flags CUDA synchronization and allocation costs without detailing optimized kernels.
Examples
Uses a 784–256–10 MLP with batches of eight and reports ten-epoch PyTorch and C timing results.
What you can do after
Diagnose training bottlenecks with stage-level timing and compare implementation correctness through epoch loss.

Read how CUDA for LLMs explains training loop →

Graph Neural Networks in Action

A training loop is an iterative procedure that processes data through a model and updates its parameters across training iterations.

How it is used
Trains graph and spatiotemporal models while monitoring validation behavior and comparing generalization.
How deep it goes
Explains initialization, forward and backward passes, validation, scheduling, and testing, without deriving optimization mathematics.
Examples
Uses parallel GCN and GraphSAGE classification plus NRI trajectory prediction with scheduled learning rates and future-horizon MSE.
What you can do after
Apply it via PyTorch to train models, track losses, tune schedules, and evaluate predictions on held-out data.

Read how Graph Neural Networks in Action explains training loop →

Also covered in

What to know first

Related concepts

Questions about training loop

What is training loop?

A training loop is an iterative procedure that processes training data, evaluates an objective, and updates model parameters.

Which Manning books explain training loop?

Deep Learning with R, Third Edition, Rearchitecting LLMs, LLMs in Production, Build a DeepSeek Model (From Scratch), Fight Fraud with Machine Learning, CUDA for LLMs and Graph Neural Networks in Action.

What should I know before learning training loop?

Manning books that teach training loop usually introduce objective function, forward pass, parameter and activation function first.

What are the parts of training loop?

Training loop includes gradient update.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.