A probability distribution is a mathematical function that maps every allowed value of a random variable to its probability. It turns uncertainty into a structured description: rather than treating data points as isolated observations, it describes where values or high-dimensional feature vectors tend to occur. In machine learning, fitting a distribution to clusters of inputs can expose structure and correlations that a yes/no classifier may hide. section 5.2 section 5.0
For discrete outcomes, the events must be exhaustive and mutually exclusive: every possible observation belongs to one event, and no observation belongs to more than one. A proper joint distribution likewise assigns probabilities to every pair of outcomes, with all table entries summing to 1. For independent events, a joint probability is the product of the individual probabilities; dependence changes that relationship. section 5.2 section 5.4 section 5.4.2
Quantized weights
Let X encode an adult's weight range: X=1 for less than 60 kg, X=2 for 60–90 kg, and X=3 for more than 90 kg. A distribution can assign P(X=1)=0.25, P(X=2)=0.50, and P(X=3)=0.25. The middle range is most likely, so repeated samples produce it most often. section 5.2 section 5.8
Table 5.2 Probability distribution for quantized weights of Statsville adults
| S1: Less than 60 kg |
S2: Between 60 and 90 kg |
S3: More than 90 kg |
|---|---|---|
| p(X = 1) = 25,000/100,000 = 0.25 | p(X = 2) = 50,000/100,000 = 0.5 | p(X = 3) = 25,000/100,000 = 0.25 |
Use the table to connect population counts, quantized outcomes, and probabilities for a complete discrete distribution. section 5.2
A distribution can be summarized by parameters and aggregate properties. Its expected value is the long-run average of repeated samples, pulled toward values with higher probability; geometrically, it is a probability-weighted centroid. Variance and covariance further describe a distribution, and standard families expose such properties through parameters. section 5.7.1 section 5.7
Sampling draws values from a distribution. In a multivariate setting, samples form a point cloud whose denser regions correspond to higher-probability regions. When the underlying distribution is unknown, sample means and variances estimate population properties; with enough samples, the sample mean approaches the expected value. section 5.8
Distribution fitting is central to generative modeling. Maximum likelihood estimates parameters that make observed training data likely, while Bayesian estimation combines a likelihood with a prior to obtain a posterior over parameters. In a variational autoencoder, an encoder emits parameters of a latent-space distribution and training samples a latent vector from it. section 13.2 section 13.11 section 14.7.1
Common deep-learning distributions can be set up, sampled, and queried for probabilities, means, variances, and covariances with PyTorch distribution calls. Formula-based probability calculations provide the mathematical counterpart, but library calls are preferred in practice. section 5.9
