The exploration–exploitation trade-off balances actions that gather information with actions expected to produce the highest known reward. Exploitation uses current Q-values; exploration tests alternatives that may reveal better policies. Choosing only one side either misses undiscovered policies or wastes useful knowledge. section 6.2.2
A practical balance is epsilon-greedy action selection: explore randomly with probability ε, otherwise choose the action with the highest expected cumulative reward, Q(W,t,a). Exploration can also decrease during learning or follow a distribution derived from Q-values. section 6.2.2
Q-learning action selection
A Q-learning implementation sets ε = 0.5, randomly selects an action when a uniform draw falls below ε, and otherwise chooses a maximizing action, breaking ties randomly. It repeats this process across generated episodes while updating Q-values.
