Standard deviation measures how widely numerical values spread around their mean. For a normal distribution, it also provides a useful scale: roughly 68% of values fall within one standard deviation of the mean. section 1.15 section 1.5
Use standard deviation to:
- Summarize spread alongside the mean, median, minimum, quantiles, and maximum. section 1.14 section 1.15
- Recognize unusually distant values by comparing them with the mean and standard deviation. section 1.9 section 3.23
- Notice the effect of extreme values: replacing a minimum with five times the maximum changes the mean and standard deviation substantially. section 1.15
Inspecting spread in a Series
Generate 100,000 normally distributed values with mean 0 and standard deviation 100, then call s.describe() to inspect count, mean, standard deviation, quantiles, and extremes. The observed standard deviation is about 100, while the mean and median remain close but not identical because the values are random. section 1.15
import math math.sqrt(((s - s.mean()) ** 2).sum() / s.count())
Compare the direct pandas calculation with the square-root formula for squared deviations from the mean. section 1.5
