liveBook · Manning

Validation loss

Also called evaluation loss

The value of a loss function computed on held-out validation data to assess a model’s predictions.

How 7 Manning books explain it

LLM Customization and Fine-Tuning

A measure of a model’s training objective on held-out data, used to assess how well it generalizes beyond its training examples.

How it is used
Monitors generalization during supervised fine-tuning and guides stopping, checkpoint selection, and hyperparameter iteration.
How deep it goes
Explains loss-curve patterns, conditional gap thresholds, and remedies, with a monitor that reads logged evaluation loss.
Examples
Illustrates loss divergence in a 450-example LoRA run and checkpoint selection in a three-epoch full fine-tuning run.
What you can do after
Diagnose training runs with loss curves and select a checkpoint or corrective action when validation loss stops improving.

Read how LLM Customization and Fine-Tuning explains validation loss →

Deep Learning with R, Third Edition

Validation loss is the loss value computed from a model's predictions on validation data.

How it is used
Monitors whether model learning remains useful on unseen validation data and helps detect overfitting during training.
How deep it goes
Defines the metric, explains strict data separation, demonstrates fit() and evaluate(), and stops short of detailed generalization theory.
Examples
Demonstrates a 30% R/Keras validation split, evaluate(), image datasets, loss curves, and val_loss-based checkpointing.
What you can do after
Operate/configure Keras training to monitor validation loss, evaluate held-out data, and retain the best-performing epoch.

Read how Deep Learning with R, Third Edition explains validation loss →

Deep Learning with PyTorch, Second Edition

Loss computed on held-out validation data.

How it is used
Uses held-out data to check whether training improvements generalize beyond observed samples.
How deep it goes
Defines the metric, compares its trend with training loss, and demonstrates epoch-by-epoch computation without detailing loss-function internals.
Examples
Uses randomized dataset splitting, separate tensors, a PyTorch training loop, and four conceptual loss-curve patterns.
What you can do after
Diagnose overfitting and generalization by monitoring training and validation losses during optimization.

Read how Deep Learning with PyTorch, Second Edition explains validation loss →

Timeless Algorithms, The Foundational Ideas

Held-out loss is the value of a loss function evaluated on observations excluded from model training.

How it is used
Measures predictive performance on held-out observations and tracks outcomes in language-model scaling experiments.
How deep it goes
Defines held-out loss and relates its decline to parameters, training data, and compute through power laws, without guaranteeing broader reliability.
Examples
Tracks next-token prediction loss as parameter counts, training-token volumes, or compute increase.
What you can do after
Evaluate scaling behavior by comparing held-out loss as model parameters, training data, or compute increase within the studied range.

Read how Timeless Algorithms, The Foundational Ideas explains validation loss →

Deep Learning with Python, Third Edition

The loss value computed on validation data to estimate how well a model generalizes beyond the training set.

How it is used
Monitors whether training improves performance on unseen data and supports selecting a model before overfitting.
How deep it goes
Explains held-out computation, Keras evaluation, curve interpretation, and checkpoint selection, but defers deeper generalization theory.
Examples
Uses shuffled NumPy splits, Keras evaluate(), image datasets, loss curves, and a ConvNet checkpoint monitored by val_loss.
What you can do after
Diagnose overfitting, compute held-out loss with Keras, and save or reload the epoch with the lowest validation loss.

Read how Deep Learning with Python, Third Edition explains validation loss →

LLMs in Production

Validation loss is the loss computed on held-out validation examples to assess a model during development.

How it is used
Uses averaged validation batches to monitor model behavior during training and development.
How deep it goes
Defines the metric and implements ten-batch averaging, but stops short of treating it as a representative performance measure.
Examples
Uses TinyStories training and validation files plus a PyTorch function that averages ten sampled validation-batch losses.
What you can do after
Evaluate model behavior during development with a repeatable validation-batch loss estimate.

Read how LLMs in Production explains validation loss →

Generative AI in Action

Validation loss is a metric measuring a model’s prediction error on examples held out for validation.

How it is used
Tracks how well fine-tuning generalizes beyond training examples during model adaptation.
How deep it goes
Defines the metric, its cost-function calculation, epoch timing, and relationship to overfitting without mathematical derivation.
Examples
Uses a GPT-3.5 Turbo fine-tuning loss graph showing loss values across training steps.
What you can do after
Evaluate fine-tuning progress and identify potential overfitting by comparing training and validation loss.

Read how Generative AI in Action explains validation loss →

Also covered in

What to know first

Related concepts

Questions about validation loss

What is validation loss?

The value of a loss function computed on held-out validation data to assess a model’s predictions.

Which Manning books explain validation loss?

LLM Customization and Fine-Tuning, Deep Learning with R, Third Edition, Deep Learning with PyTorch, Second Edition, Timeless Algorithms, The Foundational Ideas, Deep Learning with Python, Third Edition, LLMs in Production and Generative AI in Action.

What should I know before learning validation loss?

Manning books that teach validation loss usually introduce language model, objective function, overfitting and foundation model first.

What is the difference between validation loss and objective function?

The value of a loss function computed on held-out validation data to assess a model’s predictions. An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.

What is validation loss also called?

Validation loss is also called evaluation loss.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.