A Beta distribution represents uncertainty about an unknown probability, such as an ad’s click-through rate. It stays between zero and one and updates as clicks and non-clicks are observed. section 5.3
In Thompson sampling, each ad has a Beta belief. The algorithm samples a possible click-through rate for each ad, chooses the ad with the highest sample, then updates its belief using the result. section 5.3
Listing 5.6 The Thompson Sampling Agent
class ThompsonSamplingAgent:
def __init__(self, num_ads):
self.num_ads = num_ads
self.alphas = [1] * num_ads #A
self.betas = [1] * num_ads #A
def choose_action(self):
sampled_ctrs = [np.random.beta(self.alphas[i],
[CA]self.betas[i]) for i in range(self.num_ads)] #B
return np.argmax(sampled_ctrs) #C
def update(self, chosen_ad_index, reward):
if reward == 1:
self.alphas[chosen_ad_index] += 1 #D
else:
self.betas[chosen_ad_index] += 1 #E
See the agent code for Beta sampling and updates after clicks or non-clicks. section 5.3



