The probability of best arm estimates how often an arm produces the highest aggregate metric among alternatives. For click models, the metric is production CTR. Epsilon-greedy selects one apparent best arm, whereas Thompson sampling allocates traffic in proportion to these probabilities. section 3.3.1
Bootstrap sampling approximates repeated experiments from one experiment’s individual measurements. Resample each arm’s data with replacement, compute its CTR, identify the highest CTR, and count each arm’s wins. Dividing win counts by the number of bootstrap trials estimates pbest. section 3.3.1
Two click-model arms
For I_clicked = [0,0,1,0,1,1,0,0,1,0], CTR is 0.40 and its standard error is about 0.16. Generate many bootstrap CTRs for two arms, compare each pair, and estimate pbest from the fraction of trials each arm wins. Ties contribute to neither arm. section 3.3.1
