Softmax converts a vector of scores into nonnegative values that sum to one. In a neural network’s final classification layer, these values serve as probability scores for the possible classes. section 2.1
Classifying handwritten digits
A Keras model can end with a ten-unit softmax layer, producing one score for each MNIST digit class: model <- keras_model_sequential() |> layer_dense(512, activation = "relu") |> layer_dense(10, activation = "softmax"). After prediction, predictions[1, ] contains ten probabilities; which.max(predictions[1, ]) selects the most likely class, whose probability can be read directly. section 2.1
Use softmax in these practical ways:
- Interpret the final output as a distribution across mutually exclusive classes, with all class scores summing to one. section 2.1
- Choose between an explicit softmax activation and logits: Keras can normalize inside the loss when using loss_sparse_categorical_crossentropy(from_logits = TRUE). section 16.2.2
- Treat transformer output values without a softmax as logits, or unnormalized log probabilities, until exponentiation and normalization produce probabilities. section 16.2.2
Listing 2.2 The network architecture
model <- keras_model_sequential() |> layer_dense(512, activation = "relu") |> layer_dense(10, activation = "softmax")
Inspect the complete R/Keras architecture showing a dense hidden layer followed by a ten-unit softmax classification layer. section 2.1
