A probability distribution describes the possible values of a random quantity and how likely each value is. In A/B testing, each aggregate measurement is treated as one random draw from a distribution of potential measurements. section 2.3.1
Use the distribution to make an engineering decision:
- Compare aggregate execution costs for ASDAQ and BYSE. section 2.3.1
- Define delta as aggregate BYSE cost minus aggregate ASDAQ cost, then estimate its standard error. section 2.3.1
- Convert delta to z = delta / se_delta and judge whether the result is unlikely under a zero-difference assumption. section 2.3.1
Exchange-routing experiment
With ten individual measurements per exchange, compute aggregate values, standard errors, delta, and se_delta using NumPy. A sufficiently extreme z score supports routing trades toward the exchange with lower measured execution cost, while retaining a stated risk of error. section 2.3.1
