Training loop
A training loop is an iterative procedure that processes training data, evaluates a model objective, and updates model parameters to improve performance.
Deep Learning with R, Third Edition
A training loop is a repeated process that evaluates examples, adjusts model parameters, and progressively minimizes the loss function.
- How it is used
- Trains neural networks by repeatedly reducing prediction loss through weight updates on batches of data.
- How deep it goes
- Details batches, forward passes, loss, gradients, updates, and epochs, then extends to custom loops.
- Examples
- Uses MNIST digit classification and a pure-JAX linear classifier whose logged loss falls over 40 steps.
- What you can do after
- Build it from scratch with backend-native gradients, or apply it via Keras `fit()` for standard training.
Read how Deep Learning with R, Third Edition explains training loop →
LLMs in Production
A training loop is the repeated process of computing model outputs, evaluating loss, calculating gradients, and updating parameters.
- How it is used
- Organizes model training from data preparation through repeated prediction, loss, backpropagation, and parameter updates.
- How deep it goes
- Explains the loop from batches to gradients and implementation components, then extends into speed-memory tradeoffs without deriving optimization theory.
- Examples
- Uses a generative CBoW model, Hamlet tokens, a GPT built from scratch, and a speed-memory comparison table.
- What you can do after
- Build it from scratch with modular model components, batched data, loss calculation, backpropagation, and resource-aware training choices.
Graph Neural Networks in Action
A training loop is an iterative procedure that processes data through a model and updates its parameters across training iterations.
- How it is used
- Trains graph and spatiotemporal models while monitoring validation behavior and comparing generalization.
- How deep it goes
- Explains initialization, forward and backward passes, validation, scheduling, and testing, without deriving optimization mathematics.
- Examples
- Uses parallel GCN and GraphSAGE classification plus NRI trajectory prediction with scheduled learning rates and future-horizon MSE.
- What you can do after
- Apply it via PyTorch to train models, track losses, tune schedules, and evaluate predictions on held-out data.
Read how Graph Neural Networks in Action explains training loop →
Related concepts
- fit methodimplemented by · A fit method is an operation that learns or estimates model parameters, structure, or transformation statistics from supplied data.
- epochhas part · In machine learning, an epoch is one complete pass through the training data during model training.
- DataLoaderuses · PyTorch DataLoader is an iterable utility that retrieves dataset samples, optionally shuffles and transforms them, and groups them into batches.
- PyTorch Lightningimplemented by · PyTorch Lightning is a high-level framework that structures and simplifies PyTorch model, data, and training workflows.
- gradient accumulationused by · Gradient accumulation is a training technique that combines gradients from multiple smaller batches before applying an optimizer update to model parameters.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.