Objective function
An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
Advanced Algorithms and Data Structures
A loss function is a function that quantifies the discrepancy between model predictions and observed target values.
- How it is used
- Scores candidate solutions so optimization can seek low-cost graph drawings, routes, or fitted models.
- How deep it goes
- Develops definitions, cost landscapes, squared-error regression, and derivative-based graph updates; defers a fuller treatment of supervised learning.
- Examples
- Uses crossing counts, squared-error regression, force-directed drawing, and traveling-salesperson tour costs.
- What you can do after
- Evaluate/understand which objective suits a problem and whether its shape and evaluation cost favor gradient descent or simulated annealing.
Read how Advanced Algorithms and Data Structures explains objective function →
Grokking Machine Learning
Training error is a measure of a model's prediction error on the data used to train it.
- How it is used
- Measures model mistakes during training and supplies a quantity to reduce while fitting regression, classifiers, and neural networks.
- How deep it goes
- Defines errors geometrically, compares classifier objectives, and sketches gradient-based reduction; neural-network backpropagation is given as pseudocode.
- Examples
- Compares point-to-line distances, polynomial models’ training and testing errors, and four models in a selection exercise.
- What you can do after
- Evaluate/understand it by comparing candidate models’ errors and checking held-out error before choosing a model.
Read how Grokking Machine Learning explains objective function →
Math and Architectures of Deep Learning
A function assigning a scalar penalty to model parameters or predictions.
- How it is used
- Defines the quantitative target minimized while training supervised models and neural networks.
- How deep it goes
- Develops per-instance and total loss, squared-error and cross-entropy forms, loss-surface geometry, and gradient-based minimization.
- Examples
- Uses linear and parabolic losses, four-class image predictions, normalized image reconstruction, and Fast R-CNN RoI losses.
- What you can do after
- Evaluate/understand how a loss quantifies mismatch and guides parameter updates toward lower training error.
Read how Math and Architectures of Deep Learning explains objective function →
Optimization Algorithms: AI techniques for design, planning, and control problems
A loss is a scalar measure of discrepancy between a model prediction and a target that an optimization procedure seeks to reduce.
- How it is used
- Uses objectives to score candidate solutions in constrained optimization and losses to guide neural network training.
- How deep it goes
- Defines objectives and losses, works through ticket pricing, and implements average negative log likelihood for neural network parameters.
- Examples
- Uses conference ticket pricing and penguin species classification to illustrate profit maximization and training loss.
- What you can do after
- Evaluate and understand objective values when comparing feasible solutions or scoring candidate neural network parameters.
Bayesian Optimization in Action
An objective function is a function whose input is selected to maximize or minimize its output.
- How it is used
- Frames expensive black-box optimization as repeatedly querying inputs to find a maximum with few evaluations.
- How deep it goes
- Defines black-box objectives and develops one-dimensional, constrained, and four-parameter optimization examples with GP-based implementations.
- Examples
- Uses a hidden one-dimensional function, simulated airplane utility, and a constrained Forrester-function scenario with a cost function.
- What you can do after
- Apply it via BoTorch to model objective and cost observations and optimize for better feasible solutions.
Read how Bayesian Optimization in Action explains objective function →
Grokking AI Algorithms, Second Edition
Training loss is the loss computed on the examples used to fit a model.
- How it is used
- Uses training loss to measure prediction error and guide neural-network weight updates.
- How deep it goes
- Defines objectives and works through loss calculations and weight-update cycles, including mean squared error and cross-entropy.
- Examples
- Calculates loss for next-token probabilities, predicted image noise, and Q-value targets.
- What you can do after
- Evaluate and understand training progress by interpreting loss and comparing training with validation loss.
Read how Grokking AI Algorithms, Second Edition explains objective function →
Deep Learning with PyTorch, Second Edition
Loss computed on training data during fitting.
- How it is used
- Guides parameter fitting by assigning a numerical score to prediction errors on training data.
- How deep it goes
- Defines loss, compares absolute and squared error, and implements mean squared error within training and validation loops.
- Examples
- Uses temperature conversion, bird-versus-airplane classification, and diffusion noise prediction to illustrate task-specific losses.
- What you can do after
- Apply it via PyTorch to compute loss, backpropagate, update parameters, and compare training with validation loss.
Read how Deep Learning with PyTorch, Second Edition explains objective function →
Inside Deep Learning: Math, Algorithms, Models
The loss value computed during training and monitored to verify that optimization is making progress.
- How it is used
- Supplies the scalar training goal whose gradients guide model-parameter updates.
- How deep it goes
- Derives a summed training objective and walks through PyTorch updates; defers nuanced regression-loss tradeoffs.
- Examples
- Uses a negative raw-error counterexample, L1 and MSE regression losses, classification cross-entropy, and padding-aware sequence loss.
- What you can do after
- Build it from scratch as a PyTorch training objective, then monitor loss alongside separate performance metrics.
Read how Inside Deep Learning: Math, Algorithms, Models explains objective function →
Applied Reinforcement Learning
A mathematical quantity that a learning or optimization procedure seeks to minimize or maximize.
- How it is used
- Frames business optimization goals and provides differentiable targets for reinforcement learning updates.
- How deep it goes
- Develops business formulations and the expected-return objective, then compares algorithm objectives and gradient updates.
- Examples
- Uses retail inventory replenishment and a two-product bakery formulation to illustrate optimization goals.
- What you can do after
- Evaluate and understand how objectives express business goals and guide reinforcement learning updates.
Read how Applied Reinforcement Learning explains objective function →
Natural Language Processing in Action, Second Edition
Loss is a numerical measure of discrepancy between a model's prediction and a desired target, with lower values generally indicating better agreement.
- How it is used
- Uses objectives to select chatbot responses and losses to guide neural-network training.
- How deep it goes
- Explains loss choice, weighting, gradients, and optimization through a worked classifier, while deferring detailed mean squared error treatment.
- Examples
- Uses a name-label classifier with count-weighted binary cross-entropy and a chatbot response-selection pipeline.
- What you can do after
- Apply it via PyTorch by computing binary cross-entropy and using backpropagation and an optimizer to train a classifier.
Read how Natural Language Processing in Action, Second Edition explains objective function →
Grokking Machine Learning, Second Edition
A measure of the discrepancy between a regression model's predictions and target values.
- How it is used
- Measures model performance and supplies the quantity minimized during regression, perceptron, and neural-network training.
- How deep it goes
- Defines several error measures, connects them to gradient descent and stopping, and defers derivative derivations to an appendix.
- Examples
- Uses competing fitted lines, happy-and-sad classifiers, a mountain descent analogy, and an RMSE training plot.
- What you can do after
- Evaluate/understand how an error value compares models, guides parameter updates, and helps choose when training can stop.
Read how Grokking Machine Learning, Second Edition explains objective function →
Deep Learning with R, Third Edition
A loss function is a function that assigns a numerical measure of discrepancy between a model's prediction and the true target.
- How it is used
- Uses loss as the feedback signal that guides neural-network weight updates toward task targets.
- How deep it goes
- Explains loss selection and gradient-based training, then implements a weighted custom loss for object detection.
- Examples
- Uses handwritten-digit classification, temperature forecasting, and YOLO bounding-box prediction to illustrate loss choices.
- What you can do after
- Apply built-in losses through Keras and select task-appropriate loss and metric combinations.
Read how Deep Learning with R, Third Edition explains objective function →
Machine Learning System Design
A loss is a numerical value representing the discrepancy between a machine learning model's output and a target value for an instance or collection of instances.
- How it is used
- Guides training toward errors that matter for the system’s goal while separating optimization from performance assessment.
- How deep it goes
- Explains selection criteria and optimization requirements, then applies them to classification, forecasting, and loss-curve diagnosis without deriving algorithms.
- Examples
- Illustrates focal loss for imbalanced classification, quantile losses for retail forecasting, and a spam system whose loss missed the product goal.
- What you can do after
- Evaluate and select losses against optimization constraints and project goals, and diagnose suspicious training curves.
Read how Machine Learning System Design explains objective function →
Deep Learning with Python, Third Edition
A function that compares predictions with targets and returns a scalar indicating how wrong the model is for a given task.
- How it is used
- Uses loss as the training feedback signal for adjusting model weights toward task targets.
- How deep it goes
- Explains loss selection and gradient-based updates, and implements a custom multi-output detection loss.
- Examples
- Uses handwritten digit classification and YOLO bounding-box prediction to illustrate built-in and custom losses.
- What you can do after
- Apply it via Keras by choosing a task-appropriate loss and configuring it with compile().
Read how Deep Learning with Python, Third Edition explains objective function →
Sutskever's List
Training loss is the value of an objective function measuring a model's error on examples used for parameter training.
- How it is used
- Guides parameter updates during neural-network training and helps forecast gains from additional model size, data, and compute.
- How deep it goes
- Defines loss through prediction error and explains gradient-based updates, then examines scaling behavior without deriving an objective mathematically.
- Examples
- Contrasts AlexNet’s mini-batch optimization with a random-label experiment that achieves high training accuracy but near-chance test accuracy.
- What you can do after
- Evaluate/understand low training loss as an optimization result, not by itself evidence of generalization or benchmark capability.
Data Without Labels
A loss function quantifies the discrepancy between predicted and actual values for evaluating a model.
- How it is used
- Provides a quantity to minimize when assessing predictions, training neural networks, or fitting lower-dimensional representations.
- How deep it goes
- Defines loss through prediction discrepancy, then works through network weight gradients; leaves the exact stress formula outside the supplied excerpt.
- Examples
- Uses product-demand forecasts, face-identification outputs, stress goodness-of-fit values, and fuzzy-clustering convergence.
- What you can do after
- Evaluate/understand it by reading a loss score as an optimization target and tracing how weight updates seek to reduce it.
Classic Computer Science Problems in Swift: Essential techniques for practicing programmers
An error function is a mathematical function that assigns a numerical measure to the discrepancy between predicted and expected outputs.
- How it is used
- Guides A* maze search by ranking frontier states according to path cost plus estimated remaining cost.
- How deep it goes
- Defines g(n), h(n), and f(n), then shows their use in Swift node comparison without developing a prediction-error objective.
- Examples
- Uses a grid maze, step counts, and Euclidean and Manhattan distance heuristics.
- What you can do after
- Apply it via A* search to prioritize maze locations using accumulated cost and a distance heuristic.
Machine Learning Algorithms in Depth
Training loss is the value of a model's loss function evaluated on examples in its training dataset.
- How it is used
- Uses an objective to guide training or split selection, with training loss measuring prediction error on training examples.
- How deep it goes
- Derives average misclassification and mean squared error and explains CART split costs; defers cross-entropy to later treatment.
- Examples
- Works through misclassification and squared error, then uses candidate feature thresholds and an Iris-based CART implementation.
- What you can do after
- Build it from scratch to score candidate CART splits and select a feature threshold using an impurity cost.
Read how Machine Learning Algorithms in Depth explains objective function →
Statistics Every Programmer Needs
An objective function is a mathematical expression that assigns a value to each feasible decision and is maximized or minimized in an optimization problem.
- How it is used
- Frames project prioritization as maximizing feature priority while respecting budgets and strategic balance constraints.
- How deep it goes
- Explains linear formulation, binary variables, score ratios, solver negation, constraints, and result evaluation without proving optimization algorithms.
- Examples
- Uses feature value-to-effort ratios, project budgets, epic selection bounds, Python coefficients, and scipy.optimize.linprog.
- What you can do after
- Build it from scratch by defining objective coefficients, encoding constraints, solving with linprog(), and evaluating selected alternatives.
Read how Statistics Every Programmer Needs explains objective function →
Machines that Think
The criterion that determines which model parameters or outputs are favored during training.
- How it is used
- Uses objective functions to define the numerical criterion that neural networks optimize during training.
- How deep it goes
- Explains loss functions and their task-specific variants, then traces loss sensitivity through backpropagation and parameter updates.
- Examples
- Illustrates loss with regression and classification, and examines next-token prediction as an objective that favors plausible continuation.
- What you can do after
- Evaluate what a training objective rewards and whether it omits considerations such as factual verification.
LLM Evaluation and Alignment, The Foundational Ideas
An evaluation metric is a measurable quantity that represents how well an AI system satisfies a specified criterion.
- How it is used
- Uses evaluation metrics to measure desired AI behavior and guide iterative alignment.
- How deep it goes
- Explains metric selection, error analysis, and re-evaluation, while noting that a scalar score may not work as a training signal.
- Examples
- Illustrates chatbot accuracy, politeness, latency, and resolution metrics alongside BLEU and ROUGE for translation and summarization.
- What you can do after
- Evaluate/understand AI behavior by choosing criteria, interpreting metric limitations, and investigating failures before making changes.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains objective function →
LLM Customization and Fine-Tuning
A measure of how well a model’s predictions fit its training examples, typically computed from the training objective.
- How it is used
- Uses training loss to monitor fit during supervised fine-tuning, while prioritizing validation loss when judging generalization.
- How deep it goes
- Defines loss and gives diagnostic curve patterns, thresholds, and corrective actions without deriving cross-entropy.
- Examples
- Compares training and evaluation losses in a 40-example IT support LoRA run and a 450-example adapter run.
- What you can do after
- Diagnose training problems with loss curves and select the checkpoint with the lowest validation loss.
Read how LLM Customization and Fine-Tuning explains objective function →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A loss function quantifies the discrepancy between model predictions and expected target values for optimization.
- How it is used
- Defines the optimization target that gradient boosting uses to improve predictions for regression, classification, count, survival, or ranking tasks.
- How deep it goes
- Explains objective selection and gradient-based corrections, but stops short of deriving the available loss functions mathematically.
- Examples
- Uses logistic classification, insurance claim distributions, event counts, and an Airbnb NYC gradient boosting implementation with measured accuracy.
- What you can do after
- Configure XGBoost objectives for different prediction tasks and distinguish training losses from evaluation metrics.
Timeless Algorithms, The Foundational Ideas
A training objective is a quantity that a learning procedure seeks to optimize while fitting a model.
- How it is used
- Guides model fitting by specifying the quantity to improve, such as prediction loss or a constrained margin objective.
- How deep it goes
- Explains loss behavior and gradient-based updates, then formulates hard-margin SVM optimization; soft-margin treatment is deferred.
- Examples
- Compares squared-error regression, cross-entropy classification, hard-margin SVM optimization, and next-token prediction measured on held-out text.
- What you can do after
- Evaluate how a chosen objective scores predictions or constrains model fitting, and distinguish continuous loss from thresholded classification.
Read how Timeless Algorithms, The Foundational Ideas explains objective function →
Designing Deep Learning Systems: A guide for software engineers
A function that maps hyperparameter settings and their resulting training outcomes to a scalar value to be minimized or maximized.
- How it is used
- Guides HPO libraries in scoring training trials and choosing subsequent hyperparameter suggestions.
- How deep it goes
- Explains inputs, training, evaluation, returned scores, and metric reporting, but defers detailed optimization algorithms.
- Examples
- Uses Hyperopt classifier selection, Optuna regressors, and Ray Tune convolutional-network accuracy reporting.
- What you can do after
- Build it from scratch: wrap training and evaluation in a callable accepted by Hyperopt, Optuna, or Ray Tune.
Learn Generative AI with PyTorch
The objective value minimized during training, here composed of reconstruction loss and KL divergence for the VAE.
- How it is used
- Uses a combined training target to balance image reconstruction with regularization of the VAE latent space.
- How deep it goes
- Explains the objective’s two terms and its backpropagation role, but does not derive their equations.
- Examples
- Uses reconstructed human faces, 196,608-pixel inputs, 100-value latent vectors, and randomly sampled encodings for novel images.
- What you can do after
- Apply it via a VAE training loop that computes combined loss and updates encoder and decoder parameters.
Read how Learn Generative AI with PyTorch explains objective function →
Building Specialized AI Systems
Training loss is the loss value measured on the examples used to optimize a model's parameters.
- How it is used
- Guides gradient descent during fine-tuning by scoring how wrong model predictions are.
- How deep it goes
- Explains scalar error, cross-entropy, and the loss landscape, but does not develop detailed loss-function formulas.
- Examples
- Uses uncertain positive sentiment predictions, cross-entropy, a training loop, and contrasting training and validation loss.
- What you can do after
- Apply it via a training loop to measure prediction error, backpropagate gradients, update parameters, and monitor overfitting.
Read how Building Specialized AI Systems explains objective function →
Build a Text-to-Image Generator (from Scratch)
A loss function is a function that quantifies the discrepancy between a model's predictions and the desired outputs.
- How it is used
- Guides model training by measuring prediction error and specifying the discrepancy to minimize.
- How deep it goes
- Explains task-specific losses and multi-term balancing, but does not derive their mathematical formulas or optimization algorithms.
- Examples
- Uses CIFAR-10 classification, VAE image reconstruction, and DALL-E mini image-token prediction as concrete objective-function examples.
- What you can do after
- Apply it via a selected loss, optimizer, batches, and training loop for classification or generative modeling.
Read how Build a Text-to-Image Generator (from Scratch) explains objective function →
Grokking Bayes
A loss function is an objective function whose value measures an error or cost and is minimized during optimization.
- How it is used
- Guides neural-network training by turning prediction mismatch into a scalar target for gradient descent.
- How deep it goes
- Defines average squared error and parameter minimization, but defers gradient-descent mechanics to separate treatment.
- Examples
- Uses average squared error with a neural network fitted to noisy sine-function data.
- What you can do after
- Apply it via gradient descent to fit neural-network parameters using average squared error.
Inside AI
An objective function is a function that assigns a value representing the performance or desirability of a candidate solution.
- How it is used
- Scores candidate solutions in genetic algorithms, assigning fitness values that guide selection.
- How deep it goes
- Defines it as a solution evaluator that assigns fitness scores, without specifying a formula for the evaluator.
- What you can do after
- Apply a chosen objective function to score candidate solutions in a genetic algorithm.
Introduction to Generative AI, Second Edition
A loss function is a mathematical function that quantifies prediction error or undesired behavior for optimization during model training.
- How it is used
- Frames the numerical goal that guides model training across supervised, unsupervised, and reinforcement learning.
- How deep it goes
- Defines objectives and training intuition but defers the precise mathematics of updating neural-network parameters.
- Examples
- Uses minimizing errors, reducing prediction differences, maximizing rewards, and balancing mixture-of-experts training as examples.
- What you can do after
- Evaluate/understand how learning settings define goals and how those goals guide parameter updates and expert balancing.
Read how Introduction to Generative AI, Second Edition explains objective function →
Build AI Drug Discovery Pipelines
A cost function measures aggregate prediction error across multiple data instances, with lower values representing better performance.
- How it is used
- Guides weight adjustment by aggregating prediction errors across training examples.
- How deep it goes
- Defines aggregate error and works through mean squared error for linear regression.
- Examples
- Uses mean squared error to fit a solubility regression line and penalize distant residuals.
- What you can do after
- Apply mean squared error to choose linear-regression weights that fit training values.
Read how Build AI Drug Discovery Pipelines explains objective function →
Generative AI in Action
Training loss is a metric measuring the error between a model’s predictions and actual outcomes on training examples.
- How it is used
- Monitors fine-tuning progress and guides judgment about whether model predictions are improving.
- How deep it goes
- Defines training and validation loss, explains step and epoch measurements, and stops short of mathematical derivation.
- Examples
- Uses batch-level loss curves from Azure OpenAI fine-tuning and a GPT-3.5 Turbo training-loss figure.
- What you can do after
- Evaluate fine-tuning progress by comparing training and validation loss across steps and epochs.
Read how Generative AI in Action explains objective function →
LLMs in Production
A loss function is an objective that quantifies discrepancies between a model's predictions and target outputs for optimization during training.
- How it is used
- Guides weight adjustment during batched LLM training by quantifying the calculated loss after predictions.
- How deep it goes
- Explains loss within the training loop and links it to epochs, weight updates, and overfitting without deriving its formula.
- Examples
- Uses a from-scratch language-model pipeline and a GPT-style PyTorch listing as concrete training contexts.
- What you can do after
- Apply it via a training loop that batches data, obtains predictions and loss, and performs backpropagation.
Causal AI
A loss function is an objective function that quantifies the cost or error associated with model parameters or predictions.
- How it is used
- Frames parameter estimation and decision selection as optimization over error, cost, likelihood, or desirability.
- How deep it goes
- Defines loss, connects it to likelihood and regularization, and contrasts analytical solutions with numerical optimization.
- Examples
- Uses L2 parameter penalties, investment payoffs, and a utility-node diagram to make objective design concrete.
- What you can do after
- Apply likelihood-based, regularized, or utility-based objectives when fitting models or comparing uncertain decisions.
Reinforcement Learning from Human Feedback
A loss function is a scalar objective that quantifies prediction or optimization error during model training.
- How it is used
- Guides parameter updates by converting prediction, reward, or preference quality into an optimizable scalar.
- How deep it goes
- Derives policy-gradient and PPO objectives through ratios, advantages, clipping, log-probabilities, and per-token implementation.
- Examples
- Uses PPO equations, per-token log-probability ratios, policy-gradient cases, reward functions, and preference-optimization losses.
- What you can do after
- Apply it via policy-gradient or PPO losses while recognizing clipping constraints and proxy-objective failure modes.
Read how Reinforcement Learning from Human Feedback explains objective function →
Graph Neural Networks in Action
A loss function is a function that quantifies the discrepancy between a model's output and the desired target.
- How it is used
- Guides graph-model training by turning prediction errors and reconstruction goals into a value the optimizer can reduce.
- How deep it goes
- Explains computation inside training loops and composite objectives, without deriving individual loss formulas.
- Examples
- Uses masked node classification and molecular graph reconstruction with predicted adjacency, features, and QED properties.
- What you can do after
- Apply it via a GNN training loop that computes loss, propagates gradients, and updates model parameters.
Read how Graph Neural Networks in Action explains objective function →
Build an AI Agent (From Scratch)
A metric is a quantitative indicator that aggregates evaluation judgments into a value trackable across cases or over time.
- How it is used
- Uses quantitative indicators to monitor agent quality, operational behavior, regressions, and improvement over time.
- How deep it goes
- Defines rubric-based aggregation and example formulas, while stopping short of mathematical optimization or implementation code.
- Examples
- Uses pass rates for answer relevancy, citation reliability, requirement compliance, latency, cost, token use, and tool-call frequency.
- What you can do after
- Operate/configure evaluation dashboards and deployment checks that expose quality regressions through trackable indicators.
Read how Build an AI Agent (From Scratch) explains objective function →
Statistics Slam Dunk
An objective function is the mathematical function or criterion that an optimization procedure seeks to minimize or maximize.
- How it is used
- Optimizes the value of an NBA free-agent roster while honoring position, player-count, salary, and age constraints.
- How deep it goes
- Defines the objective and demonstrates its lp() implementation, but does not develop the underlying optimization mathematics.
- Examples
- Uses an NBA free-agent roster, an lp() call, selected-player output, total win shares, salary, and average age.
- What you can do after
- Apply it via lpSolve to maximize win shares for a roster that satisfies explicitly specified constraints.
AI Agents in Action, Second Edition
A product metric measures business or user outcomes associated with an agent's deployed role.
- How it is used
- Uses measurable targets to improve model predictions during training and judge deployed-agent outcomes in production.
- How deep it goes
- Explains loss, backpropagation, and metric categories, but stops short of specifying an explicit mathematical objective or optimization algorithm.
- Examples
- Uses next-token prediction, task completion, escalation, conversion, resolution, latency, grounding, and user feedback as measurable examples.
- What you can do after
- Evaluate/understand whether model training reduces prediction error and whether an agent improves its intended production outcome.
Read how AI Agents in Action, Second Edition explains objective function →
Building Reliable AI Systems
A mathematical function that quantifies prediction error to guide a model's optimization during training.
- How it is used
- Uses a training loss to measure prediction error and incorporate a penalty for demographic disparities.
- How deep it goes
- Defines cross-entropy and demonstrates a weighted fairness penalty in code, noting its possible accuracy trade-off.
- Examples
- Compares customer-service responses across demographic groups and shows a PyTorch loss function.
- What you can do after
- Apply via PyTorch a weighted demographic-gap penalty alongside cross-entropy during training.
Read how Building Reliable AI Systems explains objective function →
CUDA for LLMs
Loss is a numerical measure of the discrepancy or error that a model's training process seeks to minimize.
- How it is used
- Guides training by providing a scalar objective to minimize.
- How deep it goes
- Defines loss and gives linear-layer gradient formulas, but does not specify a loss function.
- Examples
- Uses a linear layer to show weight, input, and bias gradient calculations.
- What you can do after
- Apply matrix and reduction formulas to calculate linear-layer gradients from the loss.
Also covered in
- How Large Language Models Work
- Deep Learning with JAX
- Transfer Learning for Natural Language Processing
- Tech Startup Toolkit
- Build a Robo-Advisor with Python (From Scratch)
- Ensemble Methods for Machine Learning
- Grokking Statistics
- Knowledge Graphs and LLMs in Action
- Learn AI-Assisted Python Programming, Second Edition
- Build a Reasoning Model (From Scratch)
- The AI Pocket Book
- Fight Fraud with Machine LearningIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- parameterA parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
- gradient descentGradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
- optimizationOptimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.
Related concepts
- cross entropy lossincludes · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- mean squared errorincludes · Mean squared error is the arithmetic mean of the squared differences between corresponding predicted and reference numerical values.
- validation losscontrasts with · The value of a loss function computed on held-out validation data to assess a model’s predictions.
- simulated annealingused by · Simulated annealing is a stochastic optimization method that gradually reduces the probability of accepting worse solutions to escape local optima.
- training datasetmeasures · A training dataset is a collection of examples supplied to a learning procedure to fit or adapt a model’s parameters or behavior.
- forward passproduced by · A forward pass is a computation that propagates inputs through a model or dependency structure in the forward direction to calculate outputs or intermediate values.
Questions about objective function
What is objective function?
An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
Which Manning books explain objective function?
Advanced Algorithms and Data Structures, Grokking Machine Learning, Math and Architectures of Deep Learning, Optimization Algorithms: AI techniques for design, planning, and control problems, Bayesian Optimization in Action, Grokking AI Algorithms, Second Edition, Deep Learning with PyTorch, Second Edition, Inside Deep Learning: Math, Algorithms, Models and 34 more.
What should I know before learning objective function?
Manning books that teach objective function usually introduce language model, parameter, gradient descent and optimization first.
What is the difference between objective function and validation loss?
An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize. The value of a loss function computed on held-out validation data to assess a model’s predictions.
What kinds of objective function are there?
Kinds of objective function include cross entropy loss and mean squared error.
What is objective function also called?
Objective function is also called loss function, cost function, training loss, error function or training objective.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.









































