Likelihood measures how probable the observed data are under a candidate parameter value. Fitting a model chooses parameter values that make the observed data most likely. section 2.3.4
The estimation workflow connects likelihood to practical optimization:
- Maximize likelihood, usually by maximizing log-likelihood because it is mathematically and computationally easier. section 2.3.4
- Minimize negative log-likelihood as a loss function; add regularization when optimization should discourage extreme parameter values. section 2.3.4
- Use a derived solution for special cases such as linear regression, or numerical optimization when no closed-form solution is available. section 2.3.4
Neural-network fitting
A neural network may make exact likelihood maximization infeasible, so training settles for parameters with relatively high likelihood rather than a guaranteed optimum. section 2.3.4
