Softmax turns logits into non-negative values that sum to one, but exponentials can overflow for moderately large inputs. A maximum reduction helps prevent overflow; computing the denominator also requires a sum reduction across the vector. section 4.2
Softmax Activation Function
The softmax formula shows where exponentials and the vector-wide denominator create numerical and reduction challenges. section 4.2


