Softmax turns real-valued scores into positive weights that sum to one. It exponentiates each score and divides by the sum of the exponentiated scores. section 9.5
Softmax serves two decision roles:
- For discrete actions, it converts policy-network logits into probabilities for sampling an action. section 9.3
- For attention, it converts query-key scores into weights for combining value vectors; scaling the scores helps avoid saturation. section 9.5
Choosing a production level
For logits [1.0, 2.5, 0.5] representing Low, Medium, and High production, softmax gives probabilities about [0.17, 0.76, 0.10]. The agent can sample a level from this distribution. section 9.3
Figure 9.6 The Attention Mechanism — Query, Key, Value.
The query-key-value diagram places softmax between similarity scores and the weighted combination of values. section 9.5

























