A logit is one unnormalized score in a classifier’s score vector. Each coordinate corresponds to a class, and the largest score determines the predicted label before probabilities are computed. section 9.1.5
Logits can be any real numbers, from negative to positive infinity. Unlike probabilities, they do not need to lie between 0 and 1 or sum to 1. Softmax converts the complete logit vector into a probability vector while preserving the ordering of scores. section 9.1.5
Classifying an image
For the logit vector [9.99, 10, 0.01, −10], index 1 has the greatest score, so the predicted class is dog. Softmax produces approximately [0.497, 0.502, …], showing that dog is favored but cat remains plausible. section 9.1.5
In LeNet, fully connected layers produce a 10-dimensional logits vector, with one score for each class. A softmax layer converts these scores into probabilities, and CrossEntropyLoss compares the prediction with the ground truth. section 11.1
Equation 9.6
Use the softmax equation to connect individual logits with the normalized class-probability vector. section 9.1.5
Listing 9.4 PyTorch code for softmax
from torch.nn.functional import softmax #1 scores = torch.tensor([9.99, 10, 0.01, -10]) #2 output = softmax(scores, dim=0) #3
The PyTorch listing shows softmax applied along the score-vector dimension. section 9.1.5
