A logit is an unnormalized numerical score produced before a function converts it into a probability or normalized weight. A logit can be positive or negative; it is not itself a probability. Its meaning depends on what the model is scoring. section 11.3.2 section 12.2.1
In binary classification, an output node can calculate a logit z = v₁h₁ + v₂h₂ + b, where h₁ and h₂ are hidden-node outputs, v₁ and v₂ are weights, and b is a bias. The sigmoid function σ(z) = 1/(1 + e⁻ᶻ) converts that score into a probability. A separate decision threshold can then turn the probability into an action. section 11.3.2
Pedestrian detection
With hidden outputs 0.75 and 0.52, weights 1.2 and −0.7, and bias 0.2, the output logit is approximately 0.74. Sigmoid converts it to approximately 0.68. With a decision threshold of 0.30, the positive classification triggers braking; calculating the prediction has not yet changed the model’s parameters. section 11.3.2
In self-attention, compatibility logits are scaled query–key dot products: sᵢ = (q · kᵢ)/√dₖ, where dₖ is the key-vector dimension. Scaling limits excessively large scores. Softmax converts the scores into weights: αᵢ = eˢⁱ/Σⱼeˢʲ. The weights sum to one and determine how much each value contributes to the resulting representation. section 12.2.1
Attention and vocabulary scores
For a pronoun query, scaled logits of 0.14 for “trophy” and 0.57 for “box” become attention weights of approximately 0.39 and 0.61. The larger weight indicates stronger compatibility, not a guarantee of correct interpretation. Separately, a language model produces logits over vocabulary candidates from its final prompt representation; softmax turns those output logits into probabilities of possible continuations. section 12.2.1 section 12.2.2























