Variance measures how widely data points scatter around their mean. Small variance means values cluster together; large variance means they are more spread out. This turns a vague impression—“the data is all over the place”—into a number useful for judging predictability. section 2.17 section 2.18
Calculate variance by following the same basic recipe:
- Find the dataset’s mean. section 2.18
- Subtract the mean from each data point and square each difference. section 2.17
- Average the squared differences. For a population, divide by N; for a sample, divide by n − 1 to account for sampling uncertainty through Bessel’s correction. section 2.18
Same average, different spread
A group whose heights cluster near 6 feet has low variance. A group containing heights such as 4.5, 5, and 7 feet can have the same mean but higher variance. Higher variance generally means predictions may have larger errors and be less reliable. section 2.18
import numpy as np
temps = [77,75,68,71,60,65,81,71,56,73,
79,80,85,84,72,69,82,81,71,63,
75,83,80,80,88,76,79,73,80,81,82]
# calculate mean
mean = sum(temps) / len(temps)
# sum all the squared differences
squared_diffs = 0
for t in temps:
squared_diffs += (t - mean) ** 2
# take the average of the squared differences
variance = squared_diffs / (len(temps) - 1)
# Variance: 57.65806451612904
print("Variance: ", variance)
# From NumPy: 57.65806451612903
print("From NumPy: ", np.var(temps, ddof=1))
Inspect a temperature dataset, calculate squared differences by hand in Python, and verify the sample variance with NumPy. section 2.18
