Objective function

As explained in Designing Deep Learning Systems: A guide for software engineers

A function that maps hyperparameter settings and their resulting training outcomes to a scalar value to be minimized or maximized.

An objective function is the callable at the center of a hyperparameter optimization (HPO) trial. It receives suggested hyperparameters, trains a model with them, evaluates the completed model, and returns a numeric score for the HPO algorithm to minimize or maximize. section 5.4

An HPO workflow uses the objective function to:

• Read candidate hyperparameters from the library’s trial, args, or config object. section 5.4.1 section 5.4.2
• Run model training and evaluation for the selected configuration. section 5.4
• Return the evaluation value so the search algorithm can select later candidates. section 5.3.2
• Report intermediate metrics when the execution framework supports trial monitoring or early stopping. section 5.4.3

Hyperopt classifier search

Define objective(args) as a wrapper around train(args) and evaluate(model). Pair it with a search space containing naive Bayes, SVM, and decision-tree alternatives, then let TPE run up to ten trials to optimize model accuracy. section 5.4.1

Figure 5.11 An objective function receives the hyperparameters as input and returns a score.

Use the workflow diagram to trace hyperparameters into training and evaluation, then follow the returned score back into optimization. section 5.4

How Designing Deep Learning Systems: A guide for software engineers covers it

How deep it goes

Explains inputs, training, evaluation, returned scores, and metric reporting, but defers detailed optimization algorithms.

Examples

Uses Hyperopt classifier selection, Optuna regressors, and Ray Tune convolutional-network accuracy reporting.

What it assumes

Assumes model training and evaluation code, while introducing how libraries supply hyperparameters and consume objective scores.

What you can do after

Build it from scratch: wrap training and evaluation in a callable accepted by Hyperopt, Optuna, or Ray Tune.

How it connects

Connects hyperparameter optimization, search spaces, model training, model evaluation, trial history, and early stopping.

liveBookmanning.com

Read Designing Deep Learning Systems: A guide for software engineers on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Advanced Algorithms and Data StructuresA loss function is a function that quantifies the discrepancy between model predictions and observed target values.
• 

  liveBookmanning.com
  Grokking Machine LearningTraining error is a measure of a model's prediction error on the data used to train it.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA function assigning a scalar penalty to model parameters or predictions.
• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsA loss is a scalar measure of discrepancy between a model prediction and a target that an optimization procedure seeks to reduce.
• 

  liveBookmanning.com
  Bayesian Optimization in ActionAn objective function is a function whose input is selected to maximize or minimize its output.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionTraining loss is the loss computed on the examples used to fit a model.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionLoss computed on training data during fitting.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionLoss is a numerical measure of discrepancy between a model's prediction and a desired target, with lower values generally indicating better agreement.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA loss function is a function that assigns a numerical measure of discrepancy between a model's prediction and the true target.
• 

  liveBookmanning.com
  Machine Learning System DesignA loss is a numerical value representing the discrepancy between a machine learning model's output and a target value for an instance or collection of instances.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA function that compares predictions with targets and returns a scalar indicating how wrong the model is for a given task.
• 

  liveBookmanning.com
  Sutskever's ListTraining loss is the value of an objective function measuring a model's error on examples used for parameter training.
• 

  liveBookmanning.com
  Data Without LabelsA loss function quantifies the discrepancy between predicted and actual values for evaluating a model.
• 

  liveBookmanning.com
  Classic Computer Science Problems in Swift: Essential techniques for practicing programmersAn error function is a mathematical function that assigns a numerical measure to the discrepancy between predicted and expected outputs.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthTraining loss is the value of a model's loss function evaluated on examples in its training dataset.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsAn objective function is a mathematical expression that assigns a value to each feasible decision and is maximized or minimized in an optimization problem.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA loss function quantifies the discrepancy between model predictions and expected target values for optimization.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchThe objective value minimized during training, here composed of reconstruction loss and KL divergence for the VAE.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A loss function is a function that quantifies the discrepancy between a model's predictions and the desired outputs.
• 

  liveBookmanning.com
  Inside AIAn objective function is a function that assigns a value representing the performance or desirability of a candidate solution.
• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionA loss function is a mathematical function that quantifies prediction error or undesired behavior for optimization during model training.
• 

  liveBookmanning.com
  Generative AI in ActionTraining loss is a metric measuring the error between a model’s predictions and actual outcomes on training examples.
• 

  liveBookmanning.com
  LLMs in ProductionA loss function is an objective that quantifies discrepancies between a model's predictions and target outputs for optimization during training.
• 

  liveBookmanning.com
  Causal AIA loss function is an objective function that quantifies the cost or error associated with model parameters or predictions.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackA loss function is a scalar objective that quantifies prediction or optimization error during model training.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionA loss function is a function that quantifies the discrepancy between a model's output and the desired target.
• 

  liveBookmanning.com
  Build an AI Agent (From Scratch)A metric is a quantitative indicator that aggregates evaluation judgments into a value trackable across cases or over time.
• 

  liveBookmanning.com
  Statistics Slam DunkAn objective function is the mathematical function or criterion that an optimization procedure seeks to minimize or maximize.
• 

  liveBookmanning.com
  AI Agents in Action, Second EditionA product metric measures business or user outcomes associated with an agent's deployed role.

Compare how 30 books explain objective function →

Related concepts

• validation losscontrasts with · The value of a loss function computed on held-out validation data to assess a model’s predictions.
• mean squared errorincludes · Mean squared error is the arithmetic mean of the squared differences between corresponding predicted and target numerical values.
• simulated annealingused by · Simulated annealing is a stochastic optimization method that gradually reduces the probability of accepting worse solutions to escape local optima.
• training datasetmeasures · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.
• stressequivalent to · Stress is an MDS discrepancy measure that quantifies the difference between original dissimilarities and distances in the fitted low-dimensional representation.
• cross entropy lossincludes · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.

This explainer was generated by Manning's knowledge graph from the text of Designing Deep Learning Systems: A guide for software engineers. The section links point to the parts of the book it draws on.