Bootstrap sampling creates a simulated dataset by repeatedly drawing observations with replacement from an observed dataset. The simulated set usually has the same size as the original and approximately similar summary statistics, such as its mean and standard deviation. section 5.3.1
The method replaces repeated experiments with repeated resampling. For click-through rate (CTR), each resampled dataset produces one mean; many such means approximate the uncertainty of the observed CTR. Their standard deviation approximates the CTR’s standard error. section 3.3.1
Comparing experimental arms
Given click indicators [0,0,1,0,1,1,0,0,1,0], draw ten indices with replacement to form each bootstrap sample, then compute its mean. Repeat this for two arms, compare their sampled CTRs, and count which arm is higher. Dividing the win counts by the number of resampling rounds estimates each arm’s probability of being best. Ties are not counted in the multi-arm implementation. section 3.3.1
Building model ensembles sooner
For measurements {1, 3, 2.5, 1.5}, a bootstrap sample might be {2.5, 3, 2.5, 1.5}. Reusing logged individual measurements this way lets a recommender fit multiple prediction models without waiting for separate days of data collection. section 5.3.1
Listing 3.10 Generate a bootstrap sample from a data set
def bootstrap_sample(data):
n = len(data) #1
return data[np.random.randint(n, size=(n,))] #2
Inspect the compact NumPy implementation that samples n indices with replacement to produce an n-element bootstrap dataset. section 3.3.1
