Probability density describes how a continuous random variable is distributed across its domain. Unlike a discrete probability for one listed value, density is associated with an infinitesimally narrow interval: p(x) = lim δx→0 P(x ≤ X < x + δx). Regions with greater density produce more samples in a sample point cloud. section 5.6 section 5.10
Think of a histogram whose buckets become arbitrarily narrow. Each bucket holds the fraction of a population falling in its range; in the limit, that description becomes a probability density function. The intervals are mutually exclusive and exhaustive, so the discrete sum rule becomes an integral: integrating the density across the full domain yields 1. section 5.6 section 5.10
| The probability density function p(x) for a continuous random variable X is defined as the probability that X lies between x and x + δx with δx → 0 p(x) = limδx→0 probability(x ≤ X < x + δx) |
Use the limiting interval definition to distinguish a continuous density from a probability assigned to a discrete value. section 5.6
Probability densities supply a common language for machine-learning distributions:
- Evaluate a distribution through PyTorch functions rather than manually applying its raw formula in practice. section 5.9
- Form likelihoods by evaluating the chosen model family's density for each independent data instance. section 6.6.1
- Interpret narrow, peaked densities as concentrated sample clouds and flatter densities as more diffuse clouds. section 6.10
- Represent a VAE encoder output as a Gaussian density in latent space, then sample a latent vector during training. section 14.7.1
Density in parameter estimation
For a Gaussian model, p(x|θ) is its probability density after parameters θ specify the distribution. With independent training instances, their density evaluations form the likelihood used for parameter estimation; combining likelihood with a prior supports Bayesian estimation. section 6.6.1 section 13.7.2
