Validation loss
The value of a loss function computed on held-out validation data to assess a model’s predictions.
LLM Customization and Fine-Tuning
A measure of a model’s training objective on held-out data, used to assess how well it generalizes beyond its training examples.
- How it is used
- Monitors generalization during supervised fine-tuning and guides stopping, checkpoint selection, and hyperparameter iteration.
- How deep it goes
- Explains loss-curve patterns, conditional gap thresholds, and remedies, with a monitor that reads logged evaluation loss.
- Examples
- Illustrates loss divergence in a 450-example LoRA run and checkpoint selection in a three-epoch full fine-tuning run.
- What you can do after
- Diagnose training runs with loss curves and select a checkpoint or corrective action when validation loss stops improving.
Read how LLM Customization and Fine-Tuning explains validation loss →
Deep Learning with R, Third Edition
Validation loss is the loss value computed from a model's predictions on validation data.
- How it is used
- Monitors whether model learning remains useful on unseen validation data and helps detect overfitting during training.
- How deep it goes
- Defines the metric, explains strict data separation, demonstrates fit() and evaluate(), and stops short of detailed generalization theory.
- Examples
- Demonstrates a 30% R/Keras validation split, evaluate(), image datasets, loss curves, and val_loss-based checkpointing.
- What you can do after
- Operate/configure Keras training to monitor validation loss, evaluate held-out data, and retain the best-performing epoch.
Read how Deep Learning with R, Third Edition explains validation loss →
Deep Learning with PyTorch, Second Edition
Loss computed on held-out validation data.
- How it is used
- Uses held-out data to check whether training improvements generalize beyond observed samples.
- How deep it goes
- Defines the metric, compares its trend with training loss, and demonstrates epoch-by-epoch computation without detailing loss-function internals.
- Examples
- Uses randomized dataset splitting, separate tensors, a PyTorch training loop, and four conceptual loss-curve patterns.
- What you can do after
- Diagnose overfitting and generalization by monitoring training and validation losses during optimization.
Read how Deep Learning with PyTorch, Second Edition explains validation loss →
Timeless Algorithms, The Foundational Ideas
Held-out loss is the value of a loss function evaluated on observations excluded from model training.
- How it is used
- Measures predictive performance on held-out observations and tracks outcomes in language-model scaling experiments.
- How deep it goes
- Defines held-out loss and relates its decline to parameters, training data, and compute through power laws, without guaranteeing broader reliability.
- Examples
- Tracks next-token prediction loss as parameter counts, training-token volumes, or compute increase.
- What you can do after
- Evaluate scaling behavior by comparing held-out loss as model parameters, training data, or compute increase within the studied range.
Read how Timeless Algorithms, The Foundational Ideas explains validation loss →
Deep Learning with Python, Third Edition
The loss value computed on validation data to estimate how well a model generalizes beyond the training set.
- How it is used
- Monitors whether training improves performance on unseen data and supports selecting a model before overfitting.
- How deep it goes
- Explains held-out computation, Keras evaluation, curve interpretation, and checkpoint selection, but defers deeper generalization theory.
- Examples
- Uses shuffled NumPy splits, Keras evaluate(), image datasets, loss curves, and a ConvNet checkpoint monitored by val_loss.
- What you can do after
- Diagnose overfitting, compute held-out loss with Keras, and save or reload the epoch with the lowest validation loss.
Read how Deep Learning with Python, Third Edition explains validation loss →
LLMs in Production
Validation loss is the loss computed on held-out validation examples to assess a model during development.
- How it is used
- Uses averaged validation batches to monitor model behavior during training and development.
- How deep it goes
- Defines the metric and implements ten-batch averaging, but stops short of treating it as a representative performance measure.
- Examples
- Uses TinyStories training and validation files plus a PyTorch function that averages ten sampled validation-batch losses.
- What you can do after
- Evaluate model behavior during development with a repeatable validation-batch loss estimate.
Generative AI in Action
Validation loss is a metric measuring a model’s prediction error on examples held out for validation.
- How it is used
- Tracks how well fine-tuning generalizes beyond training examples during model adaptation.
- How deep it goes
- Defines the metric, its cost-function calculation, epoch timing, and relationship to overfitting without mathematical derivation.
- Examples
- Uses a GPT-3.5 Turbo fine-tuning loss graph showing loss values across training steps.
- What you can do after
- Evaluate fine-tuning progress and identify potential overfitting by comparing training and validation loss.
Also covered in
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
- Fight Fraud with Machine LearningIn early access (MEAP)
- Grokking AI Algorithms, Second Edition
- Hugging Face in Action
- AI-Powered Search
- Build a Large Language Model (From Scratch)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- foundation modelA foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
Related concepts
- objective functioncontrasts with · An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- generalizationmeasures · Generalization is the extension of learned patterns, abstractions, or conclusions from observed cases to new cases beyond those from which they were derived.
- overfittingmeasures · Overfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- evaluation datasetmeasures · An evaluation dataset is a collection of data or test cases used to assess a model or system against specified outcomes or criteria.
Questions about validation loss
What is validation loss?
The value of a loss function computed on held-out validation data to assess a model’s predictions.
Which Manning books explain validation loss?
LLM Customization and Fine-Tuning, Deep Learning with R, Third Edition, Deep Learning with PyTorch, Second Edition, Timeless Algorithms, The Foundational Ideas, Deep Learning with Python, Third Edition, LLMs in Production and Generative AI in Action.
What should I know before learning validation loss?
Manning books that teach validation loss usually introduce language model, objective function, overfitting and foundation model first.
What is the difference between validation loss and objective function?
The value of a loss function computed on held-out validation data to assess a model’s predictions. An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
What is validation loss also called?
Validation loss is also called evaluation loss.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






