Exploitation selects the action currently believed to produce the highest value. In Q-learning, the agent uses its accumulated Q-table knowledge instead of choosing randomly. section 11.1.1
Pure exploitation can fail when knowledge is incomplete. An epsilon-controlled policy explores randomly with some probability and otherwise chooses the action with the greatest current Q-value; epsilon decays as training progresses. section 11.1.1
Frozen Lake agent
A Frozen Lake agent begins with a zero-filled Q-table, alternates exploration and exploitation during episodes, updates values using reward and discounted future value, then exercises the learned policy with argmax. section 11.1.1
Listing 11.5 EDL_11_1_FrozenLake.ipynb: Choosing an action
def choose_action(state):
if random.uniform(0, 1) > epsilon: #1
return np.argmax(Q[state,:]) #2
else:
return env.action_space.sample() #3
Inspect the choose_action implementation that switches between random sampling and the highest-valued action. section 11.1.1
