liveBook · Manning

Softmax

As explained in Applied Reinforcement Learning

Softmax converts a vector of real-valued scores into nonnegative probabilities that sum to one.

Softmax turns real-valued scores into positive weights that sum to one. It exponentiates each score and divides by the sum of the exponentiated scores. section 9.5

Softmax serves two decision roles:

  • For discrete actions, it converts policy-network logits into probabilities for sampling an action. section 9.3
  • For attention, it converts query-key scores into weights for combining value vectors; scaling the scores helps avoid saturation. section 9.5

Choosing a production level

For logits [1.0, 2.5, 0.5] representing Low, Medium, and High production, softmax gives probabilities about [0.17, 0.76, 0.10]. The agent can sample a level from this distribution. section 9.3

Figure 9.6 The Attention Mechanism — Query, Key, Value.

The query-key-value diagram places softmax between similarity scores and the weighted combination of values. section 9.5

How Applied Reinforcement Learning covers it

How deep it goes
Defines the score normalization and works through a numerical action example and attention’s saturation concern.
Examples
Uses factory production logits and temperature-zone query-key scores as examples.
What you can do after
Apply softmax to logits or similarity scores to obtain action probabilities or normalized attention weights.
How it connects
Connects softmax to policy gradients, categorical distributions, and attention mechanisms.

Also explained in

Compare how 26 books explain softmax →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.