Standard deviation measures how widely values are distributed around their expected value. It is the square root of variance, so it expresses spread in the scale of the original values. In machine learning, spread helps distinguish concentrated data from meaningful variation and supports probability modeling. section 5.7.2
For a random variable X with mean μ, variance measures the expected squared distance from μ: var(X) = 𝔼((X − μ)²). Standard deviation is therefore √var(X). Small values indicate that observations cluster tightly around the mean; large values indicate greater dispersion. section 5.7.2
Standard deviation connects to several practical interpretations:
- In a Gaussian distribution, σ controls the bell curve’s width: increasing σ makes probability fall more slowly away from μ and produces more spread. section 5.9.2
- For multidimensional data, covariance generalizes variance by describing spread in different directions; standard deviation is the corresponding one-dimensional spread measure. section 5.7.2
- Large variation can preserve meaningful information, while small variation may be treated as noise in PCA and dimensionality reduction. section 5.7.2
Estimating Gaussian spread with PyTorch
Given observed height data X, compute sample_mean = X.mean() and sample_std = X.std(), then construct Normal(sample_mean, sample_std). The resulting Gaussian can estimate the probability that a height lies between 160 and 170 centimeters. section 6.8.1
