Standard deviation (SD) measures the typical magnitude of deviations from a quantity’s mean. For measurements with a known expectation, compute the square root of the mean squared deviations from that expectation. With black-box data, estimate the expectation with the sample mean: `sd = np.sqrt(((measurements - measurements.mean())**2).mean())`, or use `measurements.std()`. section 2.2.1
Execution-cost measurements
Three ASDAQ measurements are approximately 12.28, 10.15, and 12.62 mips. Their mean is 11.68, while individual deviations from the expectation of 12 mips reach 1.85; the mean is only 0.32 away. NumPy reports an individual-measurement SD of about 1.1. Repeating the experiment shows why aggregation helps: the SD of aggregate means falls from about 0.57 with 3 observations, to 0.18 with 30, and 0.058 with 300. section 2.2.1
Use standard deviation to quantify measurement variation and plan experiments:
- Estimate the standard error of an aggregate mean as `se = sd_1 / np.sqrt(num_ind)`, where `sd_1` is the SD of individual measurements. section 2.2.1
- Combine the individual-measurement variation from two exchanges as `sd_1_delta = np.sqrt(sd_1_asdaq**2 + sd_1_byse**2)` when designing an A/B test. section 2.3.2
- Use the estimated variation with a practical significance threshold to choose enough observations for statistical significance; the example yields 7 observations without, or 16 with, power analysis. section 2.3.2
np.random.seed(17)
measurements = np.array([trading_system("ASDAQ") for _ in range(3)])
print(measurements)
[[12.27626589 10.14537192 12.62390111]
Inspect the reproducible three-measurement NumPy example before generalizing to aggregate measurement and experiment design. section 2.2.1
