A softmax decision rule selects actions probabilistically rather than always choosing the action with the highest expected utility. Each action receives a probability proportional to its resulting utility, while a noise parameter α controls the degree of randomness. section 12.2.2
The parameter α determines how the rule behaves:
- When α = 0, the rule produces a uniform distribution over the available choices. section 12.2.2
- As α increases, action selection approaches maximizing expected utility. section 12.2.2
- Intermediate values represent agents that usually prefer higher-utility actions but do not optimize perfectly. section 12.2.2
This makes the rule useful for modeling other agents in inverse reinforcement learning, where observed behavior may be imperfect, and for balancing exploration with exploitation in bandit problems. Repeated action and outcome observations can update the model; this sequential pattern is often called Thompson sampling. section 12.2.2
Investment choices
An investor can assign nonzero probabilities to several businesses instead of committing to the currently highest-utility investment. The resulting observations provide data for revising beliefs about outcome distributions. section 12.2.2
The rule is analytically tractable and empirically validated, but it is inadmissible when the true outcome distribution is known: maximizing expected utility dominates it because softmax sometimes selects a suboptimal action. section 12.2.3
