Softmax converts several classifier scores into positive values that add up to one, so the result can be read as a probability distribution across classes. It extends the sigmoid idea from two classes to multiple classes. section 6.13 section 10.15
Start with raw scores rather than separate yes-or-no decisions. Apply the exponential function to every score, then divide each resulting value by their total. Exponentials keep every value positive while preserving the ordering of the scores. section 6.13
Dog, cat, or bird?
Suppose three classifiers produce scores of 3 for dog, 2 for cat, and −1 for bird. Exponentiating gives approximately 20.085, 7.389, and 0.368. Normalizing these values produces roughly 72% dog, 27% cat, and 1% bird. The largest score remains the most likely class, but the output also expresses uncertainty. section 6.13
Softmax is useful when:
- Several classes compete for one prediction, such as dog, cat, and bird. section 6.13
- A neural network needs one probability for each output class. section 10.15
- Classifiers need to be combined into one distribution instead of returning only a discrete label. section 6.14
In a Keras neural network, softmax can serve as the final activation for an output layer with one node per class. Categorical labels and categorical cross-entropy then support training the classifier. section 10.18

























