An objective function is a number that tells you how well a model is doing. In regression, it measures the gap between predictions and target values: a poor line gets a large number, while a better-fitting line gets a small one. It is also called an error function, loss function, or cost function. section 3.20
Figure 3.17 Two models, a bad one (on the left) and a good one (on the right). The bad one is assigned a large error, and the good one is assigned a small error.
Compare the two fitted lines: the objective function gives the bad fit a higher error and the good fit a lower error. section 3.20
For a line and a set of data points, two useful choices are absolute error and square error. Absolute error adds the vertical distances from the line to the points. Square error adds the squares of those distances. Either gives training a clear goal: make the total as small as possible. section 3.20 section 3.24
An objective function does more than score a finished model:
- It compares candidate models, assigning lower error to a model that fits the data better. section 3.20
- It guides gradient descent: repeatedly take a small step that reduces error, like carefully descending a foggy mountain. section 3.24
- It can help decide when to stop training when error reaches a target or stops decreasing much over several epochs. section 3.25
- It can balance fit against complexity by adding a regularization term to regression error. section 4.10
Choosing an error for a classifier
Counting wrong classifications is a reasonable start, but it treats every mistake alike. A more useful perceptron error can also reflect how far a misclassified point lies from the decision boundary, so a larger mistake receives a larger penalty. This matters because gradient descent needs an objective that reveals a direction for small improvements rather than one that stays flat. section 5.12 section 5.16
The same training pattern extends to neural networks: start with random weights and biases, define an error function, then repeatedly adjust the parameters to reduce it. For classification, log loss is used because predictions close to their labels receive small values and distant predictions receive large ones. section 10.9 section 10.10









































