Likelihood compares possible parameter values by how well an assumed probability model explains observed data. For fixed observations D, write L(θ) = p(D | θ): the data remain fixed while the parameter θ varies. Probability instead considers possible data under a fixed parameter. For continuous observations, the likelihood is based on a density rather than the probability of an exact value. section 3.1 section 3.1.4
Maximum likelihood estimation (MLE) selects a parameter value that maximizes L(θ). Fisher’s approach treats the parameter as fixed but unknown, rather than assigning it a prior distribution. The estimate identifies the value most compatible with the sample under the model; it does not establish the true value. section 3.0 section 10.2.2
A coin and a normal mean
If a coin lands heads seven times in 10 independent tosses with a constant probability of heads, the maximum likelihood estimate is p = 0.7. The small sample and the assumptions limit the conclusion. For independent normal observations with unknown mean and known variance, maximizing the log-likelihood instead yields the sample mean: taking logs turns the product of densities into a sum, and the maximizing value is unchanged. section 10.2.2 section 3.1.4
Likelihood also enters Bayesian inference, but it is not itself a posterior probability. Bayes’ theorem combines the likelihood P(D | H) with the prior P(H), then normalizes by the evidence P(D), to obtain P(H | D). The prior can matter greatly: a positive test with 99% sensitivity and a 1% false-positive rate gives only about a 9% disease probability when prevalence is 1 in 1,000. section 2.2.1 section 2.2.2
For independent binary outcomes yᵢ with predicted probabilities pᵢ, the likelihood is L = ∏ᵢ pᵢ^yᵢ(1 − pᵢ)^(1 − yᵢ). Each observation contributes pᵢ when yᵢ = 1 and 1 − pᵢ when yᵢ = 0. Maximizing its log is equivalent to minimizing binary cross-entropy, linking likelihood to logistic regression and classification-network training. section 3.3.2 section 11.4.2
Results depend on the model and its assumptions. For example, with independent normally distributed errors and fixed variance, maximizing the regression likelihood gives the same coefficients as minimizing squared residuals. A likelihood-based estimate should still be evaluated for how reliably it recovers the unknown parameter across repeated samples. section 3.4.2 section 3.1.5






