liveBook · Manning

Objective function

As explained in Machines that Think

The criterion that determines which model parameters or outputs are favored during training.

An objective function specifies what training should favor. In a neural network, it often takes the form of a loss function: a numerical penalty for the difference between a prediction and the desired output. Training adjusts the network’s parameters to reduce that loss across many examples. section 3.2.1 section 5.4.1

Figure 3.7 The network processes the input to produce a prediction ypred. This prediction is then compared to the expected value ytrue, and the estimation error is captured by the loss. This loss then becomes the signal that guides how the network should adjust itself during learning.

The diagram traces a prediction from the network to a loss that guides learning. section 3.2.1

The loss measures an error but does not by itself say how to change each weight. Backpropagation traces the loss backward through the network, calculating how sensitive it is to each connection. Optimizers use these sensitivities to adjust parameters toward lower loss. section 3.2.2 section 3.6

Different tasks call for different measures of error:

  • Mean squared error is used in regression and strongly penalizes large deviations from the desired value. section 3.2.1
  • Cross-entropy compares category probabilities with the correct category and heavily penalizes confident errors. section 3.2.1
  • Binary cross-entropy measures errors on yes-or-no predictions, including several independent labels. section 3.2.1

Plausibility versus truth

A language model trained to predict the next token is rewarded for matching patterns in its data, not necessarily for verifying external facts. Plausible continuation can therefore come apart from factual correctness. Evaluation or fine-tuning that rewards answers more than refusals can further favor decisive responses under uncertainty. section 6.1.2

An objective makes some differences matter numerically and leaves others outside the training signal. Unlike human learning shaped by lived consequences, a network adjusts parameters in response to an externally defined criterion. Good performance on that criterion should therefore be judged against what it measures—and what it leaves out. section 5.4.1 section 6.1.2

How Machines that Think covers it

How deep it goes
Explains loss functions and their task-specific variants, then traces loss sensitivity through backpropagation and parameter updates.
Examples
Illustrates loss with regression and classification, and examines next-token prediction as an objective that favors plausible continuation.
What you can do after
Evaluate what a training objective rewards and whether it omits considerations such as factual verification.
How it connects
Connects loss functions with backpropagation, gradients, optimization, and hallucination.

Also explained in

Compare how 42 books explain objective function →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.