chapter three
3 The algorithm of estimation: Ronald Fisher and maximum likelihood
This chapter covers
- Ronald Fisher’s On the Mathematical Foundations of Theoretical Statistics (1922) and its role in establishing rigor for statistical inference
- How maximum likelihood estimation provides a general framework for learning unknown parameters directly from observed data
- How consistency, efficiency, and sufficiency provide standards for evaluating statistical estimators
- Why Fisher rejected prior-based inference in favor of likelihood as the foundation for parameter estimation
- Why likelihood remains the underlying principle behind regression, neural network loss functions, and modern machine learning
Bayes showed how prior beliefs could be updated in light of new evidence, producing posterior probabilities that reflected both previous knowledge and observed data. Ronald A. Fisher approached the same inferential problem from a fundamentally different perspective. Rather than beginning with prior beliefs, Fisher argued that statistical inference should begin with the observed data. He argued that prior distributions were often subjective or arbitrary and therefore should not be required for scientific inference.