An objective function specifies what training should favor. In a neural network, it often takes the form of a loss function: a numerical penalty for the difference between a prediction and the desired output. Training adjusts the network’s parameters to reduce that loss across many examples. section 3.2.1 section 5.4.1
Figure 3.7 The network processes the input to produce a prediction ypred. This prediction is then compared to the expected value ytrue, and the estimation error is captured by the loss. This loss then becomes the signal that guides how the network should adjust itself during learning.

The diagram traces a prediction from the network to a loss that guides learning. section 3.2.1
The loss measures an error but does not by itself say how to change each weight. Backpropagation traces the loss backward through the network, calculating how sensitive it is to each connection. Optimizers use these sensitivities to adjust parameters toward lower loss. section 3.2.2 section 3.6
Different tasks call for different measures of error:
- Mean squared error is used in regression and strongly penalizes large deviations from the desired value. section 3.2.1
- Cross-entropy compares category probabilities with the correct category and heavily penalizes confident errors. section 3.2.1
- Binary cross-entropy measures errors on yes-or-no predictions, including several independent labels. section 3.2.1
Plausibility versus truth
A language model trained to predict the next token is rewarded for matching patterns in its data, not necessarily for verifying external facts. Plausible continuation can therefore come apart from factual correctness. Evaluation or fine-tuning that rewards answers more than refusals can further favor decisive responses under uncertainty. section 6.1.2
An objective makes some differences matter numerically and leaves others outside the training signal. Unlike human learning shaped by lived consequences, a network adjusts parameters in response to an externally defined criterion. Good performance on that criterion should therefore be judged against what it measures—and what it leaves out. section 5.4.1 section 6.1.2









































