Suppose an image might be a dog, cat, or bird. Instead of choosing one label immediately, softmax produces a probability mix such as 10% dog, 85% cat, and 5% bird. section 6.5
Softmax converts several classifier scores into positive values and normalizes them so they sum to one. It exponentiates each score, then divides by the sum of all exponentiated scores. section 6.5
Use the equation to follow the two steps: exponentiate every score, then normalize the resulting positive values. section 6.5
Apply softmax in three steps:
- Collect one score from each class classifier. section 6.5
- Exponentiate every score so all transformed values are positive. section 6.5
- Divide each transformed value by their total to obtain probabilities summing to one. section 6.5
Animal scores
For scores of 3 for dog, 2 for cat, and −1 for bird, exponentiation gives approximately 20.085, 7.389, and 0.368. Normalizing these values makes dog the most probable class while keeping every probability positive. section 6.5
Softmax in Keras
A neural network can use a two-node softmax output for two classes or a ten-node softmax output for MNIST digits. Categorical cross-entropy trains these probability outputs, and argmax selects the highest-probability label. section 10.3
